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Abstract —We study the pattern frequency vector for runs 
in probabilistic Vector Addition Systems with States (pVASS). 
Intuitively, each configuration of a given pVASS is assigned one 
of finitely many patterns, and every run can thus be seen as an 
infinite sequence of these patterns. The pattern frequency vector 
assigns to each run the limit of pattern frequencies computed 
for longer and longer prefixes of the run. If the limit does 
not exist, then the vector is undefined. We show that for one- 
counter pVASS, the pattern frequency vector is defined and takes 
one of finitely many values for almost all runs. Further, these 
values and their associated probabilities can be approximated 
up to an arbitrarily small relative error in polynomial time. For 
stable two-counter pVASS, we show the same result, but we do 
not provide any upper complexity bound. As a byproduct of 
our study, we discover counterexamples falsifying some classical 
results about stochastic Petri nets published in the 80s. 

I. Introduction 

Stochastic extensions of Petri nets are intensively used in 
performance and dependability analysis as well as reliability 
engineering and bio-informatics. They have been developed 
in the early eighties ED, El, and their token-game semantics 
yields a denumerable Markov chain. The analysis of stochastic 
Petri nets (SPNs) has primarily focused on long-run average 
behaviour. Whereas for safe nets long-run averages always 
exist and can be efficiently computed, the setting of infinite- 
state nets is much more challenging. This is a practically very 
relevant problem as, e.g., classical open queueing networks 
and biological processes typically yield nets with unbounded 
state space. The aim of this paper is to study the long- 
run average behaviour for infinite-state nets. We do so by 
considering probabilistic Vector Addition Systems with States 
(pVASS, for short), finite-state weighted automata equipped 
with a finite number of non-negative counters. A pVASS 
evolves by taking weighted rules along which any counter can 
be either incremented or decremented by one (or zero). The 
probability of performing a given enabled rule is given by 
its weight divided by the total weight of all enabled rules. 
This model is equivalent to discrete-time SPNs: a counter 
vector corresponds to the occupancy of the unbounded places 
in the net, and the bounded places are either encoded in the 
counters or in the control states. Producing a token yields an 
increment, whereas token consumption yields a decrement. 


Discrete-time SPNs describe the probabilistic branching of 
the continuous-time Markov chains determined by SPNs, and 
many properties of continuous-time SPNs can be derived 
directly from the properties of their underlying discrete-time 
SPNs. In fact, discrete-time SPNs are a model of interest in 
itself, see e.g., ED- 

Our study concentrates on long-run average pattern frequen¬ 
cies for pVASS. A configuration of a given pVASS A is a pair 
pv , where p is the current control state and v £ N“ is the vector 
of current counter values. The pattern associated to pv is a pair 
pa, where a £ {0, *} d , and a, is either 0 or *, depending 
on whether Vi is zero or positive (for example, the pattern 
associated to p( 12, 0) is p(*, 0)). Every run in A is an infinite 
sequence of configurations which determines a unique infinite 
sequence of the associated patterns. For every finite prefix 
of a run w, we can compute the frequency of each pattern 
in the prefix, and define the pattern frequency vector for w, 
denoted by F^(w), as the limit of the sequence of frequencies 
computed for longer and longer prefixes of w. If the limit does 
not exist, we put F^{w) = _L and say that F4 is not well 
defined for w. Intuitively, a pattern represents the information 
sufficient to determine the set of enabled rules (recall that 
each rule can consume at most one token from each counter). 
Hence, if we know F_a(w), we can also determine the limit 
frequency of rules fired along w. However, we can also encode 
various predicates in the finite control of A and determine the 
frequency of (or time proportion spent in) configurations in w 
satisfying the predicate. For example, we might wonder what 
is the proportion of time spent in configurations where the 
second counter is even, which can be encoded in the above 
indicated way. 

The very basic questions about the pattern frequency vector 
include the following: 

• Do we have V(F^= _L) = 0, i.e., is F4 well defined for 
almost all runs? 

• Is F4 (seen as a random variable) discrete? If so, how 
many values can F_4 take with positive probability? 

• Can we somehow compute or approximate possible val¬ 
ues of F4 and the probabilities of all runs that take these 
values? 


These fundamental questions are rather difficult for general 
pVASS. In this paper, we concentrate on the subcase of pVASS 
with one or two counters, and we also observe that with three 
or more counters, there are some new unexpected phenomena 
that make the analysis even more challenging. Still, our 
results can be seen as a basis for designing algorithms that 
analyze the long-run average behaviour in certain subclasses 
of pVASS with arbitrarily many counters (see below). The 
main “algorithmic results” of our paper can be summarized as 
follows: 

1 . For a one-counter pVASS with n control states, we show 
that Fy i is well defined and takes at most max{ 2 , 2 n— 1 } 
different values for almost all runs. These values and the as¬ 
sociated probabilities may be irrational, but can be effectively 
approximated up to an arbitrarily small relative error e > 0 in 
polynomial time. 

2. For two-counter pVASS that are stable, we show that F_ 4 
is well defined and takes only finitely many values for almost 
all runs. Further, these values and the associated probabilities 
can be effectively approximated up to an arbitrarily small 
absolute/relative error e > 0. 

Intuitively, a two-counter pVASS A is unstable if the 
changes of the counters are well-balanced so that certain 
infinite-state Markov chains used to analyze the behaviour of 
A may become null-recurrent. Except for some degenerated 
cases, this null-recurrence is not preserved under small per¬ 
turbations in transition probabilities. Hence, we can assume 
that pVASS models constructed by estimating some real-life 
probabilities are stable. Further, the analysis of null-recurrent 
Markov chains requires different methods and represents an 
almost independent task. Therefore, we decided to disregard 
unstable two-counter pVASS in this paper. Let us note that the 
problem whether a given two-counter pVASS A is (un)stable 
is decidable in exponential time. 

The above results for one-counter and stable two-counter 
pVASS are obtained by showing the following: 

(a) There are finitely many sets of configurations called 
regions, such that almost every run eventually stays in some 
region, and almost all runs that stay in the same region share 
the same well-defined value of the pattern frequency vector. 

(b) For every region R, the associated pattern frequency 
vector and the probability of reaching R can be com¬ 
puted/approximated effectively. For one-counter pVASS, we 
first identify families of regions (called zones ) that share the 
same pattern frequency vector, and then consider these zones 
rather then individual regions. 

For one-counter pVASS, we show that the total num¬ 
ber of all regions (and hence also zones) cannot exceed 
max{ 2 , 2 /t— 1 }, where n is the number of control states. To 
compute/approximate the pattern frequency vector of a given 
zone Z and the probability of staying in Z, the tail bounds of 
0 and the polynomial-time algorithm of ETl provide all the 
tools we need. 

For two-counter pVASS, we do not give an explicit bound 
on the number of regions, but we show that all regions 


are effectively semilinear (i.e., for each region there is a 
computable Presburger formula which represents the region). 
Here we repeatedly use the result of lfl8l which says that 
the reachability relation of a two-counter VASS is effectively 
semilinear. Technically, we show that every run eventually 
reaches a configuration where one or both counters become 
bounded or irrelevant (and we apply the results for one-counter 
pVASS), or a configuration of a special set C for which we 
show the existence and effective constructibility of a finite 
eager attractoQ. This is perhaps the most advanced part of 
our paper, where we need to establish new exponential tail 
bounds for certain random variables using an appropriately 
defined martingale. We believe that these tail bounds and the 
associated martingale are of broader interest, because they 
provide generic and powerful tools for quantitative analysis 
of two-counter pVASS. Hence, every run which visits C also 
visits its finite eager attractor, and the regions where the runs 
initiated in C eventually stay correspond to bottom strongly 
connected components of this attractor. For each of these 
bottom strongly connected components, we approximate the 
pattern frequency vector by employing the abstract algorithm 
of 0. 

The overall complexity of our algorithm for stable two- 
counter pVASS could be estimated by developing lower/upper 
bounds on the parameters that are used in the lemmata of 
Section[IV] Many of these parameters are “structural” (e.g., we 
consider the minimal length of a path from some configuration 
to some set of configurations). Here we miss a refinement 
of the results published in 1181 which would provide explicit 
upper bounds. Another difficulty is that we do not have any 
lower bound on \tr\ in the case when tr ^ 0, where tr is 
the mean payoff defined in Section [TV] Still, we conjecture 
that these “structural bounds” and hence also the complexity 
of our algorithm are not too high (perhaps, singly exponential 
in the size of A and in |tr|), but we leave this problem for 
future work. 

The results summarized in (a) and (b) give a reasonably 
deep understanding of the long-run behaviour of a given one- 
counter or a stable two-counter pVASS, which can be used to 
develop algorithms for other interesting problems. For exam¬ 
ple, we can decide the existence of a finite attractor for the set 
of configuration reachable from a given initial configuration, 
we can provide a sufficient condition which guarantees that 
all pattern frequency vectors taken with positive probability 
are rational, etc. An obvious question is whether these results 
can be extended to pVASS with three or more counters. The 
answer is twofold. 

I. The algorithm for stable two-dimensional pVASS pre¬ 
sented in Section [IV] in fact “reduces” the analysis of a given 
two-counter pVASS A to the analysis of several one-counter 
pVASS and the analysis of some “special” configurations of A. 
It seems that this approach can be generalized to a recursive 

'A finite eager attractor (T) for a set of configurations C is a finite set of 
configurations ACC such that the probability of reaching A from every 
configuration of C U post* (A) is equal to 1, and the probability of revisiting 
A in more than f. steps after leaving A decays (sub)exponentially in t. 
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procedure which takes a pVASS A with n counters, isolates 
certain subsets of runs whose properties can be deduced by 
analyzing pVASS with smaller number of counters, and checks 
that the remaining runs are sufficiently simple so that they can 
be analyzed directly. Thus, we would obtain a procedure for 
analyzing a subset of pVASS with n counters. 

II. In Section [V] we give an example of a three-counter 
pVASS A with strongly connected state-space whose long-run 
behaviour is undefined for almost all runs (i.e., ffi takes the _L 
value), and this property is not sensitive to small perturbations 
in transition probabilities. Since we do not provide a rigorous 
mathematical analysis of A in this paper, the above claims 
are formally just conjectures confirmed only by Monte Carlo 
simulations. Assuming that these conjectures are valid, the 
method used for two-counter pVASS is not sufficient for the 
analysis of general three-counter pVASS, i.e., there are new 
phenomena which cannot be identified by the methods used 
for two-counter pVASS. 

Related work. The problem of studying pattern frequency 
vector is directly related to the study of ergodicity properties 
in stochastic Petri nets, particularly to the study of the so- 
called firing process. A classical paper in this area Ql has 
been written by Florin & Natkin in the 80s. In the paper, 
it is claimed that if the state-space of a given stochastic 
Petri net (with arbitrarily many unbounded places) is strongly 
connected, then the firing process in ergodic. In the setting 
of (discrete-time) probabilistic Petri nets, this implies that for 
almost all runs, the limit frequency of transitions performed 
along a run is defined and takes the same value. A simple 
counterexample to this claim is shown in Fig. Q] The net 
AT has two unbounded places and strongly connected state- 
space, but the limit frequency of transitions takes two values 
with positive probability (each with probability 1/2). Note 
that Af can be translated into an equivalent pVASS A with 
two counters which is also shown in Fig. Q] Intuitively, if 
both places/counters are positive, then both of them have a 
tendency to decrease, i.e., the trend ts of the only BSCC S of 
< t ?4 is negative in both components (see Section [TI]). However, 
if we reach a configuration where the first place/counter is 
zero and the second place/counter is sufficiently large, then the 
second place/counter starts to increase , i.e., it never becomes 
zero again with some positive probability (i.e., the the mean 
payoff tr 2 is positive, where It-j is the only type II region 
of the one-counter pVASS A 2 , see Section IIVb . The first 
place/counter stays zero for most of the time, because when 
it becomes positive, it is immediately emptied with a very 
large probability. This means that the frequency of firing t 2 
will be much higher than the frequency of firing ti. When 
we reach a configuration where the first place/counter is 
large and the second place/counter is zero, the situation is 
symmetric, i.e., the frequency of firing t\ becomes much 
higher than the frequency of firing t 2 . Further, almost every 
run eventually behaves according to one of the two scenarios, 
and therefore there are two limit frequencies of transitions, 
each of which is taken with probability 1/2. This possibility 



Fig. 1: A discrete-time SPN A f and an equivalent pVASS A. 


of reversing the “global” trend of the counters after hitting 
zero in some counter was not considered in m. Further, as 
we already mentioned, we conjecture the existence of a three- 
counter pVASS A with strongly connected state-space (the 
one of Section where the limit frequency of transitions 
is undefined for almost all runs. So, we must unfortunately 
conclude that the results of m are invalid for fundamental 
reasons. On the other hand, the results achieved for one- 
counter pVASS are consistent with another paper by Florin 
& Natkin JT3] devoted to stochastic Petri nets with only one 
unbounded place and strongly connected state-space, where 
the firing process is indeed ergodic (in our terms, the pattern 
frequency vector takes only one value with probability 1). 

II. Preliminaries 

We use Z, N, N + , Q, and R to denote the set of all integers, 
non-negative integers, positive integers, rational numbers, and 
real numbers, respectively. The absolute value of a given 
x £ R is denoted by |x|. Let <5 > 0, x £ Q, and y £ R. 
We say that x approximates y up to a relative error <5, if 
either y / 0 and \x — y\/\y\ < 5, or x = y = 0. Further, 
we say that x approximates y up to an absolute error (5 
if | a; — y\ < S. We assume that rational numbers (including 
integers) are represented as fractions of binary numbers, and 
we use ||x|| to denote the size (length) of this representation. 

Let V = (V. -£), where V is a non-empty set of vertices 
and —> C V x V a total relation (i.e., for every v £ V there 
is some u £ V such that v —> u). The reflexive and transitive 
closure of —is denoted by —> *, and the reflexive, symmetric 
and transitive closure of —> is denoted by -o-*. We say that 
V is weakly connected if s t for all s,t £ V. A finite 
path in V of length k > 0 is a finite sequence of vertices 
vo,...,Vk, where Vi—>Vi+i for all 0 < i < k. The length 
of a finite path w is denoted by length (w). A run in V is 
an infinite sequence w of vertices such that every finite prefix 
of w is a finite path in V. The individual vertices of w are 
denoted by ui(0), u>(l),... The sets of all finite paths and all 
runs in V that start with a given finite path w are denoted by 
FPathv(w) and Runv(w) (or just by FPath(w) and Run[w) 
if V is understood), respectively. For a given set S C V, we 
use pre*(S) and post*(S ) to denote the set of all v £ V such 
that v —> *s and s -£ * v for some s £ S, respectively. Further, 
we say that a run w stays in S if there is a k £ N such that for 
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all £ > k we have that w(£) £ S. The set of all runs initiated 
in s that stay in S is denoted by Run(s, S). 

A strongly connected component (SCC) of V is a maximal 
subset C C V such that for all v,u £ C we have that v —»■ *u. 
A SCC C of V is a bottom SCC (BSCC) of V if for all v £ C 
and u £ V such that t-Hiwe have that u £ C. 

We assume familiarity with basic notions of probability the¬ 
ory, e.g., probability space, random variable, or the expected 
value. Given events E , F, we say that E holds for almost all 
elements of F HV(EnF) = V(F) (in particular, if V(F) = 0, 
then any event holds for almost all elements of F). As usual, 
a probability distribution over a finite or countably infinite set 
A is a function / : A —> [0,1] such that 'f2 aeA f(a) = 1. We 
call / positive if /(a) > 0 for every a £ A, and rational if 
/(a) £ Q for every a £ A. 

Definition 1 . A Markov chain is a triple M. = ( 5 , —>, Prob ) 
where S is a finite or countably infinite set of vertices, 
—> C S x S is a total transition relation, and Prob is a 
function that assigns to each state s £ S a positive probability 
distribution over the outgoing transitions of s. As usual, we 
write s —V t when s—>t and x is the probability of s—>t. 

To every s £ S' we associate the standard probability space 
(Run.M{s),F ,V) of runs starting at s, where T is the u-field 
generated by all basic cylinders Ruum{w), where w is a 
finite path starting at s, and V : F —> [ 0 , 1 ] is the unique 
probability measure such that V(Run AA {w)) = ^ 

where w(i—l)Aw(i) for every 1 < i < length(w). If 
length(w ) = 0 , we put V{RuriM{w)) = 1 . 

If A4 = (S, —> , Prob) is a strongly connected finite-state 
Markov chain, we use ps to denote the unique invariant 
distribution of A4. Recall that by the strong ergodic theorem, 
(see, e.g., ED), the limit frequency of visits to the states of 
S is defined for almost all w £ Run(s) (where s £ S is some 
initial state) and it is equal to ps- 

Definition 2. A probabilistic vector addition system with states 
(pVASS) of dimension d > 1 is a triple A = (Q , 7, W), where 
Q is a finite set of control states, 7 C Q x {—1,0, l} d x Q is 
a set of rules, and W : 7 —> N + is a weight assignment. 

In the following, we often write p4<; to denote that 
(p, k, q ) £ 7, and p q to denote that (p, K,q) £ 7 and 
W((p,K,q)) = £. The encoding size of A is denoted by ||A||, 
where the weights are encoded in binary. 

Assumption 1 . From now on (in the whole paper), we assume 
that ( Q , —>), where p^q iff p A q for some q, is weakly 
connected. Further, we also assume that for every pair of 
control states p , q there is at most one rule of the form pA q. 

The first condition of Assumption |T] is obviously safe (if 
( Q , —>) is not weakly connected, then A is a “disjoint union” 
of several independent pVASS, and we can apply our results 
to each of them separately). The second condition is also safe 
because every pVASS A can be easily transformed into another 
pVASS A' satisfying this condition in the following way: for 
each control state s of A and each rule of the form r -4 s 


we add a fresh control state sir, «] to A!. Further, for every 
s t in A we add s[r, k'] -^4 t[s, k] to A' (for all states of 
the form s[r,«/] in A'). In other words, A! is the same as A, 
but it also “remembers” the rule that was used to enter a given 
control state. 

A configuration of A is an element of conf(A) = Q x N d , 
written as pv. A rule p -4 q is enabled in a configuration 
pv if Vi > 0 for all 1 < i < d with = — 1. To A we 
associate an infinite-state Markov chain A4a whose vertices 
are the configurations of A, and the outgoing transitions of a 
configuration pv are determined as follows: 

• If no rule of 7 is enabled in pv, then pv A- pv is the only 
outgoing transition of pv, 

• otherwise, for every rule p > q enabled in pv there is a 
transition pv -^-4 q(v + n) where T is the total weight of 
all rules enabled in pv, and there are no other outgoing 
transitions of pv. 

In this paper, we also consider the underlying finite-state 
Markov chain of A, denoted by whose vertices are the 
control states of A, and p-^rq in ^4 iff p q in A and 
x = £/T p > 0, where T p is the sum of the weights of all 
outgoing rules of p in A. Note that every BSCC S of r £ A can 
be seen as a strongly connected finite-state Markov chain, and 
we use ps to denote the invariant distribution on the states 
of S. To each s £ S' we associate the vector 

changers) = ^ k ■ 
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of expected changes in counter values at s. Further, we 
define the trend of S, denoted by ts, as the vector 
ts = E se s/rs(s) • change(s). 

A pattern of A is a pair qa £ Q x {0, *} d , and the set of all 
patterns of A is denoted by Pat a- A configuration pv matches 
a pattern qa £ Pat a if p = q and for every i £ { 1 ,..., ci} we 
have that Vi = 0 or v, > 0, depending on whether a, = 0 or 
at = *, respectively. Intuitively, a pattern represents exactly 
the information which determines the set of enabled rules. For 
all w £ Rutima (pw), we define the pattern frequency vector 
: Pat a —> K. as follows: 


F A (w)(qa) 


lim #qa(w(0), ■ ■ ■ ,w(k)) 
k—>00 k + 1 


where # qa (w(0),... ,w(k)) denotes the total number of all 
indexes i such that 0 < i < k and w(i) matches the pattern 
qa. If the above limit does not exist for some qa £ Pat a, 
we put Fa(w) = _L. We say that Fa is well defined for w if 
Fa(w) f _L. Note that if Fa is well defined for w, then the 
sum of all components of Fa{w) is equal to 1. 

Let R C Run(pv) be a measurable subset of runs, and 
let e > 0. We say that a sequence (TTi, Pi),..., (H n , P n ), 
where //, : Pat a —> Q and P, £ Q, approximates the pattern 
frequencies of R up to the absolute/relative error e, if there 
are pairwise disjoint measurable subsets f?i,..., R n of R and 
vectors F±,..., F n , where Fi : Pat a —> K, such that 

• E?=i V{R i )=V{R\, 
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• F A (w) = Fi for almost all w £ Rp, 

• Hfiqa) approximates Ffiqa) up to the absolute/relative 
error e for every qa £ Pat A \ 

• Pi approximates V(Rf) up to the absolute/relative error e. 
Note that if (Hi, Pi),..., (H n ,P n ) approximates the pattern 
frequencies of R up to some absolute/relative error, then the 
pattern frequency vector is well defined for almost all w £ R 
and takes only finitely many values with positive probability. 
Also note that neither Fi,... ,F n nor Hi,..., H„ are required 
to be pairwise different. Hence, it may happen that there exist 
i ^ j such that Hi ^ Hj and Fi = Fj (or Hi = Hj and 
/•; / /•;,). 

III. Results for one-counter pVASS 

In this section we concentrate on analyzing the pattern 
frequency vector for one-dimensional pVASS. We show that 
F A is well defined and takes at most \Q\ + b distinct values 
for almost all runs, where \Q\ is the number of control states 
of A, and b is the number of BSCCs of Z' A . Moreover, these 
values as well as the associated probabilities can be efficiently 
approximated up to an arbitrarily small positive relative error. 
More precisely, our aim is to prove the following: 

Theorem 1. Let A = ( Q, 7 , W) be a one-dimensional pVASS, 
and let b be the number of BSCCs of ^ 4 . Then there is 
n < | QI +6 computable in time polynomial in ||„4|| such that for 
every e > 0, there are Hi,..., H n : Pat a —> Q computable 
in time polynomial in ||.4|| and ||e||, such that for every initial 
configuration p(k) £ conf(A) there are Pi,... ,P n £ Q 
computable in time polynomial in ||M||, |e||, and k, such that 
the sequence (Pi, Hi),..., (P n , H n ) approximates the pattern 
frequencies of Run(p(k)) up to the relative error e. 

Let us note that the “real” pattern frequency vectors Fi as well 
as the probabilities V(F A =F, ) may take irrational values, and 
they cannot be computed precisely in general. 

Remark 1. The Q | + b upper bound on n given in Theorem [7] 
is tight. To see this, realize that if \Q\ = 1, then 6=1 and 
the trivial pVASS with the only rule p-^-tp witnesses that 
the pattern frequency vector may take two different values. 
if\Q\ > 2, we have that b < \Q\ — 1. Consider a pVASS 
where Q = {p,qi ,... ,qk} and 7 contains the rules p — hp, 
p —h qi, and qi —» qi for all 1 < i < k, where all of these 
rules have the same weight equal to 1. For p( 2) as the initial 
configuration, the vector +4 takes 2k + 1 = 2 |< 3 | — 1 pairwise 
different values with positive probability. 

For the rest of this section, we fix a one-dimensional pVASS 
A = (Q, 7 - IV). We start by identifying certain (possibly 
empty) subsets of configurations called regions that satisfy the 
following properties: 

• there are at most \Q\ + b non-empty regions; 

• almost every run eventually stays in precisely one region; 

• almost all runs that stay in a given region have the same 
well defined pattern frequency vector. 

In principle, we might proceed by considering each region R 
separately and computing/approximating the associated pattern 


frequency vector and the probability of all runs that stay in R. 
However, this would lead to unnecessary technical complica¬ 
tions. Instead, we identify situations when multiple regions 
share the same pattern frequency vector, consider unions of 
such regions (called zones), and then compute/approximate the 
pattern frequency vector and the probability of staying in Z 
for each zone Z. Thus, we obtain Theorem Q] 

Technically, we distinguish among four types of regions 
determined either by a control state of A or a BSCC of Z' A . 

• Let p £ Q. A type I region determined by p is either the 
set post* (p(0)) or the empty set, depending on whether 
post*(p(0)) is a finite set satisfying post*(p(0)) C 
pre*(p( 0 )) or not, respectively. 

• Let p £ S, where S is a BSCC of 'C A . A type II region 
determined by p is either the set post* (p(0)) or the empty 
set, depending on whether post*(p( 0 )) is an infinite set 
satisfying post*(p(0)) C pre*(p( 0)) or not, respectively. 

• Let S' be a BSCC of Z A . A type III region determined 
by S consists of all p(k) £ S x N + that cannot reach a 
configuration with zero counter. 

. Let S be a BSCC of <g A , and let f? 7 (S) and R n (S) be 
the unions of all type I and all type II regions determined 
by the control states of S, respectively. Further, let D(S) 
be the set 

^SxN n pre*(Ri(S)) S j \ (ri(S) U pre*(R n (S))^ 

A type TV region determined by S is either the set D(S) 
or the empty set, depending on whether D(S) is infinite 
or finite, respectively. 

Note that if f?i, R 2 are regions of A such that Ri fl f ?2 ^ 0, 
then Ri = IF- Also observe that regions of type I, II, and III 
are closed under post*, and each such region can thus be seen 
as a Markov chain. Finally, note that every configuration of a 
type IV region can reach a configuration of a type I region, 
and the size of every type I region is bounded by |< 3| 2 (if 
R = post*(p(0)) is a type I region and p(0) —>■ *q(j), then 
j < \Q\, because otherwise the counter could be pumped to 
an arbitrarily large value; hence, |f?| < \Q\ 2 ). 

Let us note that all regions are regular in the following 
sense: We say that a set G C conf(A) of configurations 
is regular if there is a non-deterministic finite automaton A 
over the alphabet {a} such that the set of control states of A 
subsumes Q and for every configuration p(k) £ conf(A) we 
have that p(k) £ C iff the word a k is accepted by A with 
p as the initial state. If follows, e.g., from the results of m 
that if C C conf(A) is regular, then post*(C) and pre*(C) 
are also regular and the associated NFA are computable in 
time polynomial in ||A||, where A is the NFA representing C. 
Hence, all regions are effectively regular which becomes 
important in Section [IV] 

Let S' be a SCC of ( £ A . If S is not a BSCC of then 
the control states of S may determine at most |S| non-empty 
regions (of type I). If S is a BSCC of c A a , then the control 
states of S may determine at most |S| non-empty regions of 
type I or II, and at most one additional non-empty region 
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which is either of type III or of type IV (clearly, it cannot 
happen that the type III and type IV regions determined by S 
are both non-empty). Hence, the total number of non-empty 
regions cannot exceed \Q\+b, where b is the number of BSCCs 
of % j a (here we also use the assumption that ^4 is weakly 
connected). 

Now we prove that every configuration can reach some 
region in a bounded number of steps. This fact is partic¬ 
ularly important for the analysis of two-counter pVASS in 
Section HV1 

Lemma 1. Every configuration of A can reach a configuration 
of some region in at most 11|<3| 4 transitions. 

By LemmaQ] the probability of reaching (some) region from 
an arbitrary initial configuration is at least , where x m i„ 

is the least positive transition probability of A4a- This implies 
that almost every w £ Run(p(k )) visits some region R. If R 
is of type I, II, or III, then w inevitably stays in R because 
these regions are closed under post*. If R is a type IV region, 
then w either stays in R, or later visits a configuration of a 
type I region where it stays. Thus, we obtain the following: 

Lemma 2. Let p(k) be a configuration of A. Then almost 
every run initiated in p(k) eventually stays in precisely one 
region. 

As we already mentioned, computing the pattern frequency 
vector and the probability of staying in R for each region R 
separately is technically complicated. Therefore, we also in¬ 
troduce zones, which are unions of regions that are guaranteed 
to share the same pattern frequency vector. Formally, a zone 
of A is a set Z C conf(A) satisfying one of the following 
conditions (recall that ts denotes the trend of a BSCC S ): 

• Z = R, where R is a region of type I. 

• Z = R, where R is a type III region determined by a 
BSCC S of such that ts < 0. 

• Z = R, where R is a type II region determined by p £ S 
where S' is a BSCC of Zfi satisfying ts < 0. 

• Z = Rjj(S), where S is a BSCC of ^4 such that ts = 0 
and Rjj(S) is the union of all type II regions determined 
by the control states of S. 

. Z = Rii(S) U R in (S) U Riv{S), where S is a BSCC 
of T?a such that ts > 0, f?//(S) is the union of all 
type II regions determined by the control states of S, and 
Rm(S) and f?/y(S) are the type III and the type IV 
regions determined by S, respectively. 

The next two lemmata are nontrivial and represent the techni¬ 
cal core of this section (proofs can be found in Appendix [A}. 
They crucially depend on the results presented recently in 
m and m . In the proof of Lemma [3] we also characterize 
situations when some elements of pattern frequency vectors 
take irrational values. 

Lemma 3. Let p{k) be a configuration of A and Z a 
zone of A. Then F_ 4 is well defined for almost all w £ 
Run(p(k), Z), and there exists F : Pat a —> K. such that 
Fa{w) = F for almost all w £ Run(p{k),Z). Further, for 


every rational e > 0, there is a vector H : Pat a —> Q 
computable in time polynomial in ||*4|| and ||e|| such that 
H(qa) approximates F(qa) up to the relative error e for every 
qa £ Pat a- 

Lemma 4. Let p{k) be a configuration of A. Then almost 
every run initiated in p(k) eventually stays in precisely one 
zone of A. Further, for every zone Z and every rational e > 0, 
there is a P £ Q computable in time polynomial in \A\, ||e||, 
and k such that P approximates P(Run(p(k) , Z)) up to the 
relative error e. 

IV. Results for two-counter pVASS 

In this section we analyze the long-run average behavior 
of two-counter pVASS. We show that if a given two-counter 
pVASS is stable (see Definition 0 below), then the pattern 
frequency vector is well defined takes one of finitely many 
values for almost all runs. Further, these values and the 
associated probabilities can be effectively approximated up to 
an arbitrarily small positive absolute/relative error. 

Let A be a two-counter pVASS. When we say that some 
object (e.g., a number or a vector) is computable for every 
(7 £ E, where E is some set of parameters, we mean that there 
exists an algorithm which inputs the encodings of A and a, and 
outputs the object. Typically, the parameter a is some rational 
s > 0, of a pair (e,pv) where pv is a configuration. The 
parameter can also be void, which means that the algorithm 
inputs just the encoding of A. 

A semilinear constraint p i s a function p : Q —y ‘I 1 , 
where $ is the set of all formulae of Presburger arithmetic 
with two free variables x, y. Each p determines a semilinear 
set [</?] C conf(A) consisting of all p(v\,vf) such that 
p[p)[x/v\, y/v 2 \ is a valid formula. Since the reachability 
relation —> * of A is effectively semilinear |[T 8 l and semilinear 
sets are closed under complement and union, all of the sets 
of configurations we work with (such as C[R .-\, R/f\ defined 
below) are effectively semilinear, i.e., the associated semilin¬ 
ear constraint is computable. In particular, the membership 
problem for these sets is decidable. 

Given pv £ conf(A) and D C conf(A), we use 
Run(jpv —>* D) to denote the set of all w £ Run(pv) that 
visit a configuration of D, and Run(pv -fy* D) to denote the 
set Run(pv) \ Run(pv —>■* D). Note that if D = 0, then 
Run(pv fy* D) = Run(pv). 

Intuitively, our aim is to prove that the set C = conf(A) 
is “good” in the sense that there is a computable n £ N 
such that for every rational e > 0 , there exists a computable 
sequence of rational vectors H 1 ,... ,F[ n such that for every 
pv £ C, there are computable rational P\,... ,P n such that 
the sequence (Pi, -Hi),..., (P n , H n ) that approximates the 
pattern frequencies of Run(pv) up to the absolute/relative 
error e. This is achieved by first showing that certain simple 
subsets of configurations are good, and then (repeatedly) 
demonstrating that more complicated subsets are also good 
because they can be “reduced” to simpler subsets that are 
already known to be good. Thus, we eventually prove that 
the whole set conf(A) is good. 


6 


For our purposes, it is convenient to parameterize the notion 
of a “good” subset C by another subset of “dangerous” 
configurations D so that the above conditions are required 
to hold only for those runs that do not visit D. Further, we 
require that every configuration of C can avoid visiting D with 
some positive probability which is bounded away from zero. 

Definition 3. Let A = (Q,y,W) be a pVASS of dimen¬ 
sion two, and let C, D C conf(A). We say that C is good 
for D if the following conditions are satisfied: 

• There is 5 > 0 such that V(Run(pv —>* D )) < 1 — 6 for 
every pv G C. 

• There is a computable n G N such that for every 

£ > 0, there are computable H \,..., H n : Pat a —> Q 
such that for every pv € C there are 

computable Ppv.i, ■ ■ ■, Ppv,n G Q such that 

(Ppv, l) H i),..., ( Ppv,m H n ) approximate the pattern 
frequencies of Run(pv -ft* D) up to the absolute 
error e. 

Note that in Definition 0 we require that 
(Ppv.i, H i),..., (Ppv,m Hn) approximate the pattern 
frequencies of Run(pv -ft* D) up to the absolute error e. As 
we shall see, we can always compute a lower bound for each 
positive Ppv : i and II,, which implies that if P pv , and Hi can 
be effectively approximated up to an arbitrarily small absolute 
error e > 0 , they can also be effectively approximated up to 
an arbitrarily small relative error e > 0 . 

The next definition and lemma explain what we mean by 
reducing the analysis of runs initiated in configurations of C 
to the analysis of runs initiated in “simpler” configurations of 
Ci,...,C k . 

Definition 4. Let A be a pVASS of dimension tw>o, C C 
conf(A), and £ = {C \,..., Ck} a set of pairwise disjoint 
subsets of conf(A). We say that C is reducible to £ if for 
every e > 0 , there are computable semilinear constraints 
ipi ,..., ipk such that 

• [^i] Q Ci f or every 1 <i <k; 

• for all 1 < i < k and pv G [v 5 *]* we have that 
V(Run(pv —>* Di )) < e, where D t = U;/i Cj- 

• for every pv G C and every 5 > 0, there is a computable 
< G N such that the probability of reaching a configura¬ 
tion of \pf\ U • • • U in at most £ transitions is at 
least 1 — 5. 

Lemma 5 . If C is reducible to £ = {C ±,..., Ck} and every 
Ci is good for Di = (j . Cj, then C is good for 0. 

Proof: For every 1 < % < k, let n, be the computable 
constant for Ci which exists by Definition [3] The constant n 
for C is defined as n = n i- Now let us fix some e > 0. 
Since C is reducible to {C i,..., Ck}, there are computable 
constraints ,..., <pk such that, for every l < i < k, 
we have that \ip.j\ C Ci and V(Run(piVi —>* Di)) < e/4 
for every p,V{ G [v^]- Further, there are computable 
H it i ,..., H i>ni : Pat A ->• Q such that for every p i v l G 
there are computable P PiVi ,i, ■ ■ ■, Pp iVi ,ni G Q such that 


(PpiVi,!, Hip ),..., (P PiVi ,ni,H iini ) approximate the pattern 
frequencies of Run(piVi -fir* Df) up to the absolute error e/4. 
Now let pv G C. Then there is a computable £ G N such that 
the probability of reaching a configuration of [<^i] U • • • U [</?&] 
in at most £ transitions is at least 1 — e/4. Hence, we 
can effectively construct a finite tree T rooted by pv which 
represents the (unfolding of) the part of Ml a reachable from 
pv. A branch in this tree is terminated when a configuration of 
[y>i] U • • • U is visited, or when the length of the branch 
reaches £. For every 1 < i < k, let L, be the set of all leafs a 
of T labeled by configurations of J^]. We use P a to denote 
the (rational and computable) probability of reaching a from 
the root of T, and label (a) to denote the configuration which 
is the label of a. For every 1 < i < k and every 1 < j < rii, 
we put Ppv,,,j = Jf aeLi Pa ■ Piabei(a),j■ It is straightforward 
to verify that the sequence 

(Ppv, 1,1 , H\ i),..., (Ppv,i,m, H\ ni ), 

(Ppv,2,1, H 2 .I ), . ■ ■ , (Ppv ,l,n±, ^ 2 ,U 2 

(Ppv,k, 1 , Hk, 1) , ■ ■ ■ , (Ppv,h,nk , Hk,nk ) 

approximates the pattern frequencies of Run(pv) up to the 
absolute error e. In particular, realize that almost every 
w G Run(pv) eventually “decides” for some Ci, i.e., there 
is in G N such that w(m) G Ci and for all m! > m we 
have w(m') f Di (this is where we use the first condition 
of Definition [3j. Hence, the pattern frequency vector is well 
defined and approximated up to the absolute error e/4 by some 
of the above Hi j for almost all w G Run(pv). ■ 

For the rest of this section, we fix a two-counter pVASS 
A = (Q, 1 ,W) (recall that A satisfies Assumption QJ- For 
i G {1, 2}, we define a one-counter pVASS At = (Q, 7 i, Wf) 
and a labeling Lj 17; —> {—1,0,1} as follows: s t in A, 
and Li((s, n(i),t)) = n( 3 —i) iff s -^ 4 -1 in A. Note that A, is 
obtained by “preserving” the i-th counter; the change of the 
other counter is encoded in Li. Also observe that Af, 'Ci,, 
and A'a 2 are the same Markov chains. 

The results of Section [III] are applicable to A\ and .42- 
Let R be a type II or a type IV region of A,. We claim 
that there is a unique tr G R such that for almost all runs 
w G Run(p(k), R), where p(k) G R, we have that the limit 

,. E"4o L i(mle(w,j)) 

lim — - - 

n—>- 00 n 

exists and it is equal to tj. j (here, rule(w,j )) is the unique 
rule of 7 i which determines the transition w(j) —> w(j+l)\ 
cf. Assumption Q}. In other words, is the unique mean 
payoff determined by the labeling L, associated to R. To see 
this, consider the trend ts of the associated BSCC S of Yj’a- 
If R is a type IV region, then tr = ts(3—i) for almost all 
w G Run(p(k),R) (in particular, note that if ts(i) < 0 then 
V(Run(p(k), R)) = 0 ; see Section [TIB. If ts(i) > 0 and R 
is a type II region, then tr — ts(3—i), because the frequency 
of visits to configurations with zero counter is zero for almost 
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all w £ Run(p(k)), where p(k) £ R (see @). Finally, if 
t s (i) < 0 and R is a type II region, then R is ergodic because 
the mean recurrence time in every configuration of R is finite 
0 , and hence tr takes the same value for almost all w £ 
Run(p(k), R), where p(k) £ R. 

Although the value of tr may be irrational when R is 
of type II and ts{i) < 0, there exists a formula <V(x) of 
Tarski algebra with a fixed alternation depth of quantifiers 
computable in polynomial time such that $[x/c] is valid iff 
c = tr. Hence, the problem whether tr is zero (or positive, or 
negative) is decidable in exponential time 03; and if tr < 0 
(or tr > 0 ), there is a computable ieQ such that x < 0 (or 
x > 0 ) and |x| < |tr|. 

Definition 5. Let A = [Q, r y,W) be a pVASS of dimen¬ 
sion two. We say that A is stable if the following conditions 
are satisfied: 

• Let S be a BSCC of ^4 such that the type IV region 
determined by S is non-empty in A\ or Ai, or there is 
p £ S such that the type II region determined by p(0) 
is non-empty in Ai or A 2 - Then the trend tg is non-zero 
in both components. 

• Let R by a type II region in Ai such that ts{i ) < 0, where 
S is the BSCC of&A associated to R. Then tr 7 ^ 0. 

Note that the problem whether a given two-counter pVASS A 
is stable is decidable in exponential time. Our aim is to prove 
the following theorem: 

Theorem 2. Let A = (Q,J,W) be a stable pVASS of 
dimension two. Then the set conf(A) is good for 0. 

For the rest of this section, we fix a pVASS A of dimension 
two a present a sequence of observations that imply Theo- 
rem[2] Note that A is not necessarily stable, i.e., the presented 
observations are valid for general two-dimensional pVASS. 
The stability condition is used to rule out some problematic 
subcases that are not covered by these observations. 

In our constructions, we need to consider the following 
subsets of configurations: 

« C[Ri,R- 2 \. where f?i € Reg(Ai) and f ? 2 € Reg(A 2 ), is 
the set of all p(mi,mi) £ conf(A) such that p{m\) £ 
Ri and p(m 2 ) £ R 2 ', 

• B[b ], where b £ N, consists of all pv £ conf(A) such 
that for every qu £ post*(pv) we have that «( 1 ) <6 or 
u( 2 ) < 6 ; 

• Cs[ci~b\ Ac 2 «6 2 ], where S CQ, b±, b 2 £ N, and 

are numerical comparisons (such as = or <) consists of 
all p(mi, to 2 ) £ conf(A) such that p £ S, m\ ~ 61 , and 
m 2 ~ b -2 . Trivial constraints of the form c; > 0 can be 
omitted. For example, Cq [ci = 0 A c 2 > 6] is the set of 
all q(0,m) £ conf(A) where m > 6 , and Cs[c\ < 2 ] 
is the set of all q(n,m) £ conf(A) where q £ S and 
n < 2 . 

• Zs, where S C Q, consists of all p(mi,m 2 ) such that 
p £ S and some counter is zero (i.e., mi = 0 or m 2 = 0). 

• £’s[ 6 i, 6 2 ], where S C Q and b\, 6 2 £ N, consists of 
all p(mi,m 2 ) such that p £ S, some counter is zero. 


and every q , (ni,n 2 ) £ post* {p{m\, m 2 )) satisfies the 
following: 

- if rii = 0 , then n 2 < 6 2 ; 

- if n 2 = 0 , then m < b\. 

Note that all of these sets are semilinear and the associated 
semilinear constraints are computable. 

A direct consequence of Lemma |T] is the following: 

Lemma 6 . Let b = 11|Q| 4 , and let £ be a set consisting of 
B[b] and all C[R\, f? 2 ] where R\ £ Reg(Ai), i ? 2 £ Reg{A 2 ). 
Then conf(A) is reducible to £. 

To prove Lemma [ 6 ] it suffices to realize that there is a 
computable k £ N such that every pv £ conf(A) can 
reach a configuration of some C[f?i,i? 2 ] or B[b] in at most 
k transitions. 

Hence, it suffices to prove that B[b\ and all C[If , /f 2 ] are 

good for 0. All cases except for those where R\ and f ? 2 are 

of type II or type IV follow almost immediately. To handle 
the remaining cases, we need to develop new tools, which we 
present now. We start by introducing some notation. 

Given a finite path or a run w in A4j, and £ £ N, where 
I < lengthfw), we denote by x^\w), x^\w), and p^\w) 
the value of the first counter, the value of the second counter, 
and the control state of the configuration w{t), respectively. 
Further, T(w) denotes either the least £ such that x^\w) = 0, 
or 00 if there is no such £. For every i £ N, [pv -P* qu, i] 
denotes the probability of all w £ Run(pv) such that 
T(w) > i, w(i) = qu, and w(j) qu for all 0 < j < i. By 

[pv —>* qu] = Y^rLo\P v we denote the probability 

of reaching qu from pv before time T. We also put 

OO 

[pv ->* q{ 0 , *), i] = ^[pv ->* q{ 0 , k), i] 
k =0 

and 

OO 

[pv -P* q{ 0 , *)] = ->* q( 0 , k)] . 

k =0 

For a measurable function X over the runs of Af. 4 , we use 
Ep„[X] to denote the expected value of X over Run(pv). 

The following theorems are at the very core of our analysis, 
and represent new non-trivial quantitative bounds obtained by 
designing and analyzing a suitable martingale. Proofs can be 
found in |f6l . 


Theorem 3 . Let S be a BSCC of ^4 such that ts( 2) < 0, 
and let R be a type II region of _4 2 determined by some state 
of S. Then there are rational ai,bi > 0 and 0 < z \ < 1 
computable in polynomial space such that the following holds 
for all p{ 0) £ R, n £ N, and i £ N + : 

'P P (n,o)(T < 00 A 4 T) > i) < cn-zl 1 " 1 . 


Moreover, if V p t n fi)(T < 00) = 1, then 


E, 


'p(n, 0) 


AT) 


< 


b\ 

a 1 • z 1 

1 - 4 1 


In particular, none of the bounds depends on n. 
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Theorem 4. Let S be a BSCC of Ajt such that tg( 2) < 0, 
and let R be a type II region determined by some state of 
S such that tr > 0. Then there are rational a ,2 , 62 > 0 and 
0 < Z 2 < 1 computable in polynomial space such that for all 
configurations p(n, 0), where p(0) £ R, and all q £ Q, the 
following holds: 

[p(n, 0 ) —>* q( 0 , *)] < n ■ a 2 ■ zf ' b2 

Theorem 5. Let R be a type II region of A 2 such that tr < 0. 
Then there are rational 03 , 63,^3 > 0 and 0 < Z 3 < 1 com¬ 
putable in polynomial space such that for all configurations 
pin, 0), where p(0) £ R, and all q £ Q, the following holds 
for all i > where H is computable in polynomial space: 

[p(n, 0 )->*</((),*),*] < 

The above theorems are use to prove that certain configu¬ 
rations are eagerly attracted by certain sets of configurations 
in the following sense: 

Definition 6 . Let C,D C conf (A). We say that pv £ C is 
eagerly attracted by D ifV(Run(pv —> *D)) = 1 and there are 
computable constants a, z £ Q, £ £ N, and k £ N + (possibly 
dependent on pv), where a > 0 and 0 < z < 1 , such that for 
every I' > I, the probability of visiting D from pv in at most 

krjj 

(! transitions is at least 1 — a ■ z . Further, we say that C is 
eagerly attracted by D if all configurations of C are eagerly 
attracted by D, and D is a finite eager attractor if D is finite 
and post*{D ) is eagerly attracted by D. 

Markov chains with finite eager attractors were studied 
in H]|. The only subtle difference is that in (l), the probability 
of revisiting the attractor in at most i transitions is at least 
1 — zf. However, all arguments of m are valid also for the 
sub-exponential bound 1-a-z ^ adopted in Definition [ 6 ] (note 
that some quantitative bounds given in m, such as the bound 
on K in Lemma 5.1 of ID, need to be slightly adjusted to 
accommodate the sub-exponential bound). In QX it was shown 
that various limit properties of Markov chains with finite eager 
attractors can be effectively approximated up to an arbitrarily 
small absolute error £ > 0. A direct consequence of these 
results is the following: 

Proposition 1. Let D C conf (A) be a finite eager attractor. 
Then D is good for 0. 

Let us also formulate one simple consequence of Theorem [4] 

Corollary 1. For every BSCC S of Zfi we have the following: 

• If tg is negative in some component, then every configu¬ 
ration pv where p £ S is eagerly attracted by Zg. 

• If both components of tg are positive, then for every 
£ > 0 there is a computable b e such that for every 
configuration pv where p £ S' and v > (b e ,b e ) we have 
that V(Run(pv^t* Zg)) < e. 

The following theorem follows from the results about one- 
counter pVASS presented in 0 . 


Theorem 6 . For every BSCC S of ho a we have the following: 

• If tg is negative in some component, then every configu¬ 
ration pv where p £ S is eagerly attracted by Zg. 

• If both components of tg are positive, then for every 
e > 0 there is a computable b s such that for every 
configuration pv where p £ S and v > (b e ,b e ) we have 
that V(Run(pv —>■* Zg)) < e. 

In the next lemmata, we reduce the study of pattern frequencies 
for certain runs in A4a to the study of pattern frequencies for 
runs in one-counter pVASS (i.e., to the results of Section HIIli. 
This is possible because in each of these cases, one of 
the counters is either bounded or irrelevant. Proofs of the 
following lemmata are straightforward. 

Lemma 7. For every b £ N, the set B[b} is good for 0. 

Lemma 8 . The set C[R\, R 2 ], where R\ or R 2 is a type I or 
a type III region, is good for 0. 

So, it remains to consider sets of the form C[R.\. R 2 ], where 
the regions Ii \, If are of type II or type IV. We start with 
the simple case when the trend ts of the associated BSCC is 
positive in both components. 

Lemma 9. Let C[i?i,i? 2 ] be a set such that f?i,f ?2 are 
regions of type II or type TV, and the trend tg of the associated 
BSCC S of^A is positive in both components. Then C[R\, Rf\ 
is good for 0 . 

Proof: Let b £ N be a bound such that for every pv £ 
conf (A) where p £ S and v > {b,b) we have that there 
exists a “pumpable path” of the form pv —>■ *p(v+u) where u 
is positive in both components. Note that such a b exists and 
it is computable (in fact, one can give an explicit upper bound 
on b in the size of S ; see, e.g., 0). 

By Lemma 0 B[b] is good for 0. We show that 
Cg[c\ > b A C 2 > b] is good for B[b\. By our choice of b and 
Theorem[ 6 ] there is <5 > 0 such that V(Run(pv -fA -B[ 6 ])) > 5 
for every pv £ Cg [ci > b A C 2 > b]. Further, almost all runs of 
Run(pv ft * B[b\) have the same pattern frequency vector Fg 
where Fg{q {*, *)) = ps(q) for all q £ S, and Fg(a ) = 0 for 
the other patterns. 

Now we prove that , If\ is reducible to 

{B[b\,Cg[c 1 > b A C 2 > 6 ]}. By Theorem [ 6 ] we obtain 
that for every e > 0 there is a computable b e such that 
for every configuration of qu where u > (b e , b £ ) we 
have that V(Run(qu *Zg)) < e. Let <p be a semilinear 
constraint where <p(s) = x>b+b e A y>b+b e for all 

s £ S, and <p(s) = false for all s £ Q \ S. Then 
M C Cs[ci > b A C 2 > 6 ] and for every qu £ [(/?] we 
have that V(Run(qu —> *B[b])) < £. Further, there exists 
a computable k £ N such that every configuration of 
C[Ri, R 2 ] can reach a configuration of B[b\ U [ 92 ] in at most 
k transitions. This implies that for every 6 > 0, there is a 
computable t £ N such that every configuration of C[R±, R 2 ] 
reaches a configuration of B[b] U [</?] in at most t steps with 
probability at least 1 — <5. ■ 
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To prove Theorem [2] it suffices to show that the following 
sets of configurations are good for 0, where we disregard the 
subcases ruled out by the stability condition. In particular, due 
to Lemma [7] we can safely assume that some component of 
ts is negative. 

(a) C[Ri 7 R 2 ], where both If and R 2 are of type II. 

(b) C\R\ 7 R 2 ], where Ri is of type IV and R 2 is of type II, 
or Ri is of type II and R 2 is of type IV. 

(c) C[R ,-[, f? 2 ], where both If and R 2 are of type IV. 

The most interesting (and technically demanding) is the 
following subcase of Case (a). Here we only sketch the main 
ideas, a full proof can be found in |6l . 

Lemma 10. Let C[R\,R 2 \ be a set of configurations where 
Ri and R 2 are of type II, fs(2) < 0, tr 1 < 0, and tr 2 < 0. 
Then C[R\, R 2 \ is good for 0. 

Proof Sketch: Let C be the set of all configurations of the 
form q(0 7 m ) G C[Ri,R 2 \ satisfying m < (a\ •zj 1 )/(l — z^ 1 ), 
where a\ , b\ , z\ are the computable constants of Theorem [3] 
We prove that C[R\ 7 R 2 ] is eagerly attracted by C. This 
immediately implies that C is a finite eager attractor, hence C 
is good for 0 by PropositionQ] We also immediately obtain that 
C[Ri,R 2 ] is reducible to {C}, which means that C[Ri 7 R 2 \ 
is good for 0 by Lemma [5] 

Let pv G C[Ri,R 2 ]. Since ts( 2) < 0 and tr, < 1, almost 
every run w G Run(pv) eventually visits a configuration of 
Cs[c 2 = 0], and, from that moment on, visits configurations 
of both Cs [c 2 = 0] and Cs [c\ = 0] infinitely often. 

Denote by 0o (w) the least I such that w(£) G Cs[c 2 = 0]. 
Given k > 1, denote by @fc(u>) the least I > Qk-i(w ) such 
that the following holds: 

• If k is odd, then w(£) G Cs[c\ = 0]. 

• If k is even, then w(£) G Cs[c 2 = 0]. 

We use Theorems[6][2 and[5]to show that there are computable 
constants a > 0 and 0 < z < 1 such that for all k > 0 and all 
(GNwe have that 

r pv {Qk-Qk-i>£) < d-(z)^ 

Here 0_i = 0. Observe that 0o is the sum of the number 
of transitions needed to visit Zs for the first time (the first 
phase) and the number of transitions need to reach Cs [c 2 = 0] 
subsequently (the second phase). Due to Theorem [6] the 
probability that the first phase takes more than £ transitions is 
bounded by a - z e for some computable a > 0 and 0 < 2 < 1. 
Note that the length of the second phase depends on the value 
of c 2 after the first phase. However, the probability that this 
value will be larger than £ can be bounded by a ■ z ( as well. 
Finally, assuming that the first phase ends in a confi guration 
<7(0, m), Theorem 0 gives a bound a' ■ [z')^ l ~ m on the 
probability of reaching Cs[c 2 = 0] in at least £ transitions. By 
combining these bounds appropriately, we obtain the above 
bound on 0o- 

Now let us consider ©& — Qk-i for k > 0. Let us assume 
that k is even (the other case follows similarly). The only 
difference from the previous consideration (for 0 q) is that 


now the first phase consists of the part of the run up to the 
0 / i: _i -th configuration, and the second phase from there up 
to the 0fc-th configuration. Using Theorem [3] and induction 
hypothesis, we derive a bound a ■ z e on the probability that 
the height of the second counter in the 0/-_i -th configuration 
will be at least £. Then, as above, we combine this bound with 
the bound on the probability of reaching Cs [c 2 = 0] in £ steps 
from a fixed configuration of Cs [ci = 0], 

In order to finish the proof, we observe that the probability 
of reaching a configuration of C between the 0;, : _ i -th and 
0 ^-th configuration is bounded away from zero by a com¬ 
putable constant. This follows immediately from Theorem [3] 
which basically bounds the expected value of c 2 in the 
th configuration. Denoting by Rounds (w) the least number 
k such that w(Qk(w)) G C, we may easily show that 
V pv (Rounds > £) < c t for a computable constant 0 < c < 1. 

Finally, we combine the bound on the number of rounds 
(i.e., the bound on V pv (Rounds > £)) with the bound on the 
length of each round (i.e., the bound on ’P p „(0fc —©fc-1 > £)), 
and thus obtain the desired bound on the number of steps to 
visit C. ■ 

For the other cases (inch Cases (b) and (c)), we show that 
the set of configurations C we aim to analyze is eagerly 
attracted by computable semilinear sets of configurations 
Ci,.... L4, where each Ci is either good for 0 or good 
f0r U Cj. In all these cases, it is easy to see that the 
configurations of C reach a configuration of ljf=i with 
probability one, and the argument that C is eagerly attracted 
Ui=i (- 'i ' s a simplified version of the proof of Lemma [To] 
(in some cases, the proof is substantially simpler than the one 
of Lemma ITOl) . Therefore, in these cases we just list the sets 
Ci...., Cfi and add some intuitive comments which explain 
possible behaviour of the runs initiated in configurations of C. 

When defining the aforementioned sets C\,...,Ck, we 
use the following computable constants Bjj 7 Biv, Du G N, 
which are numbers (not necessarily the least ones) satisfying 
the following conditions: 

• if p(0) G R, where R is a type IV region of A, for some 
i G {1,2}, then p( 0) can reach a configuration a type I 
region in at most Bjv transitions. 

• if p( 0) G R , where R is a type II region of Ai such that 
ts(i) < 0 and tr > 0, then there is a finite path w from 
p(0) to p(0) of length smaller than Bjj such that the total 
Li -reward of all transitions executed in w is positive. 

• for every pv G conf(A) and every i G {1, 2}, if v(i) = 0, 
v(3— i) > Djj, and p(0) G R for some type II region 
of Ai such that either ts(i) > 0 and ts(3—i) < 0, or 
ts(i) < 0 and tr < 0, then there exists qu G post*(pv) 
such that u(i) > ma x{Bjj 7 Biv} and u(3 —i) = 0. 

The existence and computability of Bn, Biy, and Du 
follows from simple observations about the transition structure 
of A4a (these constants are in fact small and their size can 
be explicitly bounded in ||^4||). 

Lemma 11. For all to, n G N and a BSCC S of^A such that 
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ts is negative in some component, the set E s [m, n] is good 
for 0. 

Proof: From the definition of Es [m, n] and Theorem [6] 
we immediately obtain that Es [m, n] is a finite eager attractor 
(even if Es[m,n] = 0). Hence, the claim follows from 
Proposition Q] ■ 

Now we consider the remaining subcases of Case (a). 

Lemma 12. Let C[R\. R.fi be a set of configurations where 
Ri and R 2 are of type II, ts{ 2) < 0, £s(l) > 0, and tr 2 > 0. 
Then C[R \, Rf\ is good for 0. 

Proof: Let £ = {E[B H , Du], Cs[c 2 =0 A ci>Bjj]}. 
Observe that E[Bjj, Djj] is good for 0 by Lemma fTTI We 
show that Cs[c2=0 A c\>Bu] is good for E[Bu, Du] and 
that C[R\, R 2 ] reducible to £. Hence, C[R\, Rf\ is good for 0 
by Lemma [5] 

To see that Cs[c2=0 A c\>Bu] is good for E[Bu,Djj], 
realize that for every pv £ Cs[c2=0 A cf>Bu] we have 
that almost all runs of Run(pv) that do not visit a con¬ 
figuration of E[B[ [. D[ f eventually behave as if the first 
counter did not exist, which means that the long-run be¬ 
haviour of almost all of these runs is the same as the 
behavior of the runs of A 2 initiated in p(0) (here we also 
use the defining property of Du). Further, it follows from 
the definition of Bjj and Theorem [4] that there exists a 
<5 > 0 such that V(Run(pv -fy* E[Bu,Du\)) > S for every 
pv £ Cs[c2=0 A C\>Bjj], 

By Theorem 0] for every e > 0 there exists a computable 
semilinear constraint <p such that [<p] C Cs[c 2=0 A cf>Bu] 
and for every qu £ [<p] we have that the probability of visiting 
C[c\=Bjj] (and hence also E[Bjj, Djj]) is bounded by e. 

Now let pv £ C[f?i,f? 2 ] and <5 > 0. We need to show 
that there is a computable ( £ N such that the probability 
of reaching a configuration of E[Bu, Du] U [<p] in at most 
I transitions is at least 1 — <5. Since t§(2) < 0, every pv £ 
C[Ri, R 2 ] is eagerly attracted by Z$. Similarly as in the proof 
of Lemma flOl we show that almost every run visits Cs [cy =0] 
infinitely many times, and that the probability that the length 
between two consecutive visits to Cs[c 2 = 0 ] exceeds l decays 
sub-exponentially in l. Further, the probability of vising a 
configuration of E[Bjj, Du] U [ 95 ] from a configuration of 
Cs[c 2 = 0 ] is bounded away from zero by a fixed constant. 
Hence, we can argue as in the proof of Lemma [TOl ■ 

Lemma 13. Let C'[f?i,f? 2 ] be a set of configurations where 
Ri and R 2 are of type II, ts( 2) < 0, £s( 1) < 0, tr, > 0, 
and tr 2 > 0. Then C[R\, R 2 ] is good for 0. 

Proof: Let £ be the set consiting of E[Bjj, Bjj], 
Cs [c2=0 A ci >Bu], and Cs [ci=0 A cf>Bu]. Clearly, each 
C £ £ is either good for 0 or good for the union of 
all sets in £ \ {C} (see Lemma 0T] and the proof of 
Lemma 1 1 21 . For every e > 0, there are computable semilin¬ 
ear constraint pi,p 2 such that [<pi] C Cs[c2=0 A C\>Bu], 
[</ ? 2 ] Q Cs[ci=0 A C 2 >Bu] satisfying the requirements of 
Definition 0] Note that there is a k £ N such that for every 


configuration of Zs there is a finite path of length at most k 
to a configuration of E[Bu, Bjj] U [<pi] U [y^]- The rest of 
the argument is even simpler than in Lemma [12] ■ 

Lemma 14. Let C'[f?i,f?2] be a set of configurations where 
Ri and R 2 are of type II, ts{ 2) < 0, t-s{ 1) < 0, tr, < 0, 
and tr 2 > 0. Then C'[f?i,f?2] is good for 0. 

Proof: Let £ = {E[Bu, Du], C's[c2=0 A ci>£?//]}. We 
show that C[R\, R>\ reducible to £ similarly as in Lemma021 

■ 

The case when R\ and R 2 are of type II, £5(2) < 0, 
£s(l) < 0, tr 1 > 0, and tr 2 < 0 is symmetric to the case 
considered in Lemma IT4l 

Now we continue with Case (b) 

Lemma 15. Let C[f?i,f?2] be a set of configurations where 
Ri is of type IV and R 2 is of type II such that ts{ 2) < 0 and 
tr 2 > 0. Then C[f?i,f?2] is good for 0. 

Proof: Let £ be the set consisting of E[Bjj, Bjy], 
Cs[c2=0 A c\>Bu], and all C[R' 1 ,R2], where Ii\ is a type I 
region reachable from R\ in A\. We show that C'[f?i,f?2] 
reducible to £ similarly as in previous lemmata. ■ 

Lemma 16. Let C\R \, Ii/ 2 ] be a set of configurations where 
R\ is of type IV and R 2 is of type II such that ts{ 2) < 0 and 
tr 2 < 0. Then C[f?i,f?2] is good for 0. 

Proof: Let £ be the set consisting of E[Djj , Bjy] an d a H 
C[R[,R 2], where R\ is a type I region reachable from If in 
Ai- Then C[f?i,f?2] reducible to £ and each C £ £ is good 
for 0. ■ 

Lemma 17. Let C'[f?i,f?2] be a set of configurations where 
R\ is of type II and R 2 is of type IV such that £s(2) < 0 and 
£5(1) > 0. Then C[R\,R 2 ] is good for 0. 

Proof: Let £ be the set consisting of E{Biv, Djj] and 
all C\lf . Iff, where R ’. 2 is a type I region reachable from R 2 
in A 2 . Then C\R.\. R/f\ reducible to £. Further, all elements 
of £ are good for 0. ■ 

Note that the case when R\ is of type II and is of type IV 
such that £g(2) < 0 and ts{ 1) < 0 is symmetric to the cases 
covered in Lemma 05] and Lemma 06] 

Finally, in the next lemma we consider Case (c). 

Lemma 18. Let C[f?i,f?2] be a set of configurations where 
both R\ and R 2 are type IV regions, and the trend ts of the 
associated BSCC S of is negative in some component. 
Then C[R±,R 2 ] is good for 0. 

Proof: Let £ be the set consisting of E\Ifv■ Brv] and 
all C[Ri,R' 2 \, CfR] , R 2 ], C[R' 1 ,R' 2 ], where R[ is a type I 
region reachable from Ri in Ai (for i £ {1,2}). We show 
that C]R\ , R 2 ] reducible to £. ■ 

V. SOME NOTES ON THREE-COUNTER PVASS 

In this section we give an example of a 3-dimensional 
pVASS A such that A4 y \ is strongly connected, and the 
pattern frequency vector seems to take the J_ value with 
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(2,0,0);P, (0,2,0);P, (0,0,2);P, (-1,-1,-1);R 



Fig. 2: A 3-dimensional pVASS A. For suitable weights 
P. Q. R > 0, we have that Fa = -L almost surely. 

probability one (this intuition is confirmed by Monte Carlo 
simulations, see below). Further, the example is insensitive 
to small changes in rule weights, and it also shows that 
the method of Section [IV] based on constructing pVASS of 
smaller dimension by “forgetting” one of the counters and then 
studying the “trend” of this counter in the smaller pVASS is 
insufficient for three (or more) counters. 

The pVASS A is shown in Fig. [2] Some rules increase the 
counter by more that 1, so these should be formally replaced 
by several rules using auxiliary control states. Intuitively, A 
behaves in the following way. Suppose we start in an initial 
configuration p(m, 0,0), where m is “large”. Then, A starts 
to decrease the first counter and increase the second one. On 
average, the value of the second counter becomes 2 m when the 
first counter is decreased to zero, and the third counter is kept 
“small”. So, “on average” we eventually reach a configuration 
p( 0, 2to, 0) in about 2m transitions. Then, the second counter 
is decreased and the third counter is increased, where the value 
is again doubled “on average”, using 4m transitions. Thus, we 
reach a configuration p(0. 0,4m). Then, we “pump” the tokens 
from the third counter to the first one, reaching p( 8 m, 0, 0) in 
about 8 m transitions. And so on. Observe that the fc-th phase 
takes about 2 k transitions, and so at the end of each phase, 
about half of the time was spent in configurations with the 
“current” pattern. Hence, the pattern frequency oscillates. 

A precise formulation of this phenomenon, and a formal 
proof that almost all runs really behave in the above indicated 
way, are technically demanding and we do not provide them in 
this paper. For the reader’s convenience, we have implemented 
a simple Maple sheet which can be used to perform Monte 
Carlo simulations of A and observe the above described 
phenomenon in practice^. 

Note that the oscillation of A is insensitive to small changes 
in rule weights. However, if we modify A into A! so that the 
counter value is decreased on average in each phase (e.g., we 
start in p(m, 0,0), and then reach p{ 0 , m— 1 , 0), p(0, 0 , m—2 ), 
p(m — 3,0, 0), etc., on average), then the sum of the counters 
has a tendency to decrease and A4 A' has a finite attractor. 
This means that the pattern frequency vector is well defined 
for almost all runs of A!. Still, the behaviour of all two-counter 

2 Available at http://www.cs.ox.ac.uk/people/stefan.kiefer/pVASS-simulation.txt 


machines B\, 82, 83 obtained from A by “forgetting” the first, 
the second, and the third counter, is essentially similar to the 
behaviour of B[. B ' 2 - and B ' 3 obtained from A! in the same 
way (for example, both in B\ and B[, the second counter has a 
tendency to increase and the third has a tendency to decrease). 
Hence, we cannot distinguish between the behaviour of A and 
A! just by studying the “trends” in the two-counter pVASS 
obtained by “forgetting” one of the counters. This indicates 
that the study of 3-dimensional pVASS requires different (and 
perhaps more advanced) methods than those presented in this 
paper. 
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Appendix A 
Proofs of SectionHITI 

Lemma 19. Let R 7 ^ 0 be a type II region. Then every configuration of pre* (R) can reach a configuration of R in at most 
4|Q | 3 transitions. 

Proof: Let R = post* (p(0)) be a type II region. Recall that R is strongly connected. For every i > 0, let L, = {(/ £ Q 
q{i) £ posf*(p(0))}. We start by showing that there is r < \Q\ such that C L i+T for every i £ N, and if i > \Q\ 2 , then 
Li = L i+T . Hence, the structure of R is “ultimately periodic” and the period r is small. 

Let r be the least j > 0 such that p £ Lj. We claim that r < \Q\. Since R is infinite, all L, are non-empty, and hence 
there are 0 < i < j < \Q\ such that Li D Lj f 0. Let r £ Li D Lj. Then r(i) —> *p( 0), hence also r(j) —y *p(j—i), which 
means p £ Lj_ j. Now we show that Li C L i+T for every i £ N; to see this, first realize that p(0) —>*p(r). If r £ L j, then 
p(0) —> *r(i), and hence also p(r) —> *r(r+z). This means that r(r+i) is reachable from p( 0), i.e., r £ L i+r . It remains to 
prove that if i > \Q\ 2 , then L t = L i+T . Clearly, there is k < \Q\ such that L k . T = L^ k+ 1 y r . Since k ■ r < \Q\ 2 , it suffices 
show that for every i > k ■ t we have that Li D Li +T . Let s £ Li +T , and let r £ L( k+1 y T . Then r((fc+l) • t) — > *s(i+r), and 
this finite path inevitably contains a suffix which is a finite path from t((k+l) • r) to s(i+r) such that t £ L( k+ 1 y T and the 
counter is never decreased below (fc+1) • r along this suffix. Hence, there is also a finite path from t(k ■ r) to s(i), and since 
t £ L k -T, we obtain s £ Li. 

Now let q(k) £ pre*{R ), and let w be a path of minimal length from q(k) to a configuration of R. Suppose that the last 
configuration of w is 7 '(to) £ R. 

First we show that w cannot contain a subpath of the form t(i) —7 *s(j ) where i — j > \Q\ 2 . Suppose that w contains such 
a subpath. Then w also contains a subpath of the form v(n+£r) —> *v{n), where l > 1, which can be safely removed from 
w and the suffix of w after the configuration v(n) can be trivially adjusted so that the resulting path w' leads from q(k) to 
r(m+fr). Since r(m+fr) £ R (see above), we obtain a contradiction with our choice of w. 

Further, we prove that the counter stays bounded by k + 3\Q\ 2 in every configuration visited by w. Suppose the converse. 
Then w contains a subpath of the form t(k+2\Q\ 2 ) —> *s(fc+3|(5| 2 ). By applying the observation of the previous paragraph, 
we obtain that the counter stays above k + \Q\ 2 in all configurations visited by w after f(fc+2|Q| 2 ), and above k + 2\Q\ 2 
in all configurations visited by w after s(fc+3|(5| 2 ). In particular, the last configuration r(m) of w satisfies m > k + 2|Q| 2 . 
Further, the subpath t(k+2\Q\ 2 ) —> *s(fc+3|(3| 2 ) must contain a subpath of the form v(n) —> *v(n+£r) where 1 < £ < \Q\. If 
we delete this subpath from w and adjust the configurations visited after v(n+£r), we obtain a path w' from q{k) to r(m— £t). 
Since r(m) £ R, m > k + 2|<5| 2 , and £t < \Q\ 2 , we obtain that r(m— £t) £ R (see above). Thus, we obtain a contradiction 
with our choice of w. 

To sum up, w can visit at most -11 Q \ 3 different configurations, and hence its length cannot exceed 4|Q| 3 . ■ 

Lemma |T| Every configuration of A can reach a configuration of some region in at most 11|Q | 4 transitions. 

Proof: We start with three auxiliary observations. Let S' be a BSCC of and let Hi (S) and Rjj(S) be the unions of 
all type I regions and all type II regions determined by all q £ S, respectively. Further, let Rjjj(S) be the type III region 
determined by S. We have the following: 

(a) If Rin(S) ^ 0, then for all q £ S and all £ > \Q\ we have that q(£) £ Rjjj(S). 

(b) Let D$ be the set 

SxN n pre*(Ri(S)) S j \ (^hiS) U pre* {R H (S)) 

If Dg contains a configuration q(£) where i > 4|Q | 3 + \Q\ 2 , then Dg is infinite (i.e., the type IV region determined by 

S is exactly Dg). 

(c) If q(£) £ pre*(Ri(S)), then q{£) can reach a configuration of f?/(S) in at most £\Q\ + 4|Q | 3 transitions. 

Observation (a) follows by observing that if q{£) —> *s(0) where r, s £ S and £ > |Q|, than the path from q(£) to s(0) contains 
a subpath of the form t(i) —>■ *t(j), where i > j. This means that every configuration of S x N can reach a configuration with 
zero counter, because S is strongly connected. 

To prove Observation (b), assume that Dg contains a configuration q{£) where £ > 4|Q| 3 + \Q \ 2 and Dg is finite. Then 
there is the largest £' such that q{£') £ Dg. Obviously, £' > 4|Q| 3 + \Q\ 2 . We show that q{£') £ pre*(Rn(S)), which is 
a contradiction. Recall that every non-empty type II region determined by a control state of S is ultimately periodic and its 
period r is bounded by \Q\ (see the proof of Lemma ITbli. Let k be the product of the periods of all non-empty type II regions 
determined by the control states of S. Then q(£'+n) £ pre*(Rn(S)) (otherwise, we have a contradiction with the maximality 
of £’). Hence, q{£'-\- k) can reach a configuration v(m) of some type II region in at most 4|Q| 3 transitions, which means 
that m > k + \Q\ 2 , and the configuration q{£') can reach the configuration v(m—K). By our choice of k and the fact that 
m—K > \Q\ 2 . we obtain that v{m—rt) belongs to the same type II region as v(m) (see the proof of Lemma [l9l). 
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Observation (c) is obtained in two steps. We show that 

(A) q{£) can reach (some) configuration with zero counter in at most £\Q\ + \Q \ 2 transitions; 

(B) if s(0) —»*t(0) where s,t £ S, then s(0) can reach f(0) in at most |Q| 3 + \Q\ transitions. 

Note that Observation (c) follows immediately from (A) and (B). To prove (A), we distinguish two possibilities. If there is a 
decreasing cycle, i.e., a path of length less than \Q\ of the form t(i) —» *t(j ) where j < i, then q{£) needs at most \Q\ — 1 
transitions to reach a configuration with zero counter or a configuration t(£ + c) where c < \Q\. In the second case, at most 
(£ + c)(|Q| — 1) < t\Q\ + \Q\ 2 — |Q| transitions are needed to reach a configuration with zero counter from t{l + c), and 
hence q{£) can reach a configuration with zero counter in at most £\Q\ + \Q\ 2 transitions. If there is no decreasing cycle, then 
q(£) can still reach a configuration with zero counter (because q{£) £ pre* (f?/(S))), and hence there is a path w of minimal 
length from q[t) to a configuration with zero counter. It follows easily that if the length of this path exceeds £\Q\, then w 
either contains a decreasing cycle or can be shortened. 

To prove part (B), consider a path w of minimal length from s(0) to t(0). One can easily show that the counter value must 

be bounded by \Q \ 2 along because w could be shortened otherwise. Hence, w can visit at most Q | 3 + |Q| configuration, 

which means that its length is bounded by \Q \ 3 + \Q\. 

Now we can finish the proof of Lemma Q] Let p(k) be a configuration of A. If p(k) £ pre*(R) for some type II region R, 
then p(k) can reach R in at most 4|Q| 3 transitions by Lemma [T9l Otherwise, let us first consider the case when p{k) —»• *t(i) 
for some configuration t(i) where i > \Q\. Then such a t(i) is reachable from p(k) in at most \Q \ 2 transitions. Further, t(i) 
can reach a configuration r(j), where r £ S for some BSCC S of Z’a, in at most \Q\ transitions. If r(j ) £ f?/(S), we are 
done. Otherwise, if r(j) can reach a configuration of Rjjj(S), then such a configuration is reachable from r(j) in at most 
|<51 2 transitions (here we use Observation (a)). Otherwise, r(j) £ Ds . If j > 4|Q| 3 + \Q\ 2 , then D$ is the type IV region 
determined by S by Observation (b) and we are done. If j < 4|Q| 3 + |Q| 2 , then r(j) can reach a configuration of Ri(S) in at 
most 91 Q| 4 transitions by Observation (c). Hence, p(k) can reach a configuration of some region in at most 11|Q| 4 transitions. 

It remains to consider the case when p{k) cannot reach a configuration t(i) such that i > \Q\. Then the total number of 
configurations reachable from p(k) is bounded by \Q\ 2 . Each of these configurations is reachable in at most \Q \ 2 transitions, 
and some of them must belong to a type I or a type III region. ■ 

Before proving the next lemmata, we need to introduce some notation. For all configurations p(k) and q{£), we use 
Run(p(k) —»■ *q(£)) to denote the set of all w £ Run(p(k)) that visit q(£). We also use Run(p(k), f) to denote the set 
of all w £ Run(p(k)) such that the counter stays positive in some suffix of w, and Run(p(k)As)’ where S is a BSCC of 
to denote those w £ Run(p(k)A) which visit a configuration with control state in S. For all p,q £ Q, we use [p-lq] to 
denote the probability of all w £ Run(p( 1)) that visit q( 0) and the counter stays positive in all configurations preceding this 
visit. Finally, we use [pf] to denote 1 — W- 1 ?]- 

Lemma |3] Let pik) be a configuration of A and Z a zone of A. Then F _4 is well defined for almost all w £ Run(p(k ), Z), 
and there exists F : Pat a —> R such that F_a{w) = F for almost all w £ Run(p(k ), Z). Further, for every rational e > 0, 
there is a vector F£ : Pat a —> Q computable in time polynomial in ||_4.|| and ||e|| such that FI(qa) approximates F(qa) up to 
the relative error e for every qa £ Pat a. 

Proof: For every BSCC S of ^4, we define a vector F$ : Pat a —> Q as follows: Fs(q( 0)) = 0 for all q £ Q, 
F s {q{*)) = 0 for all q £ Q \ S, and Fs(q(*)) = ps(q ) f° r all q £ S (recall that ps is the invariant distribution of S ). Note 
that Fs is a rational vector that can be computed in time polynomial in ||Ll||. 

Let Z be a zone of A. If Z = 0, then the claim follows trivially (according to the definitions adopted in Section [III F can 
be chosen arbitrarily, and we can put H = F). Now let Z be a non-empty zone. We proceed by considering possible forms 
of Z. 

Let us first assume that Z = R, where R is a type I region. Then R can be seen as a strongly connected Markov chain 
with at most \Q \ 2 vertices (see the remarks before Lemma [T9]>. and the corresponding invariant distribution // /-> is computable 
in time polynomial in ||^4||. Hence, for almost all w £ Run(p(k),R) we have that Fa{w) = Fr, where Fji(q(0)) = pn(q( 0)) 
for all q( 0) £ R, F R (q( 0)) = 0 for all q( 0) £ R, and F R (q(*)) = Y^ q (k)eR,k>o for a H Q G Q (the empty sum is 

equal to 0). 

If Z = R where R is a type III region determined by a BSCC S of Zf such that ts < 0, then the actual counter value 
does not influence the limit behaviour of runs staying in R, which means that Fa{w) = Fs for almost all w £ Run(p(k), R). 

If Z = Rii(S) U Rih(S) U Riv(S), where S is a BSCC of If a such that ts > 0, then almost all runs of Run{p{k ), Z)) 
are diverging, i.e., for every £ £ N and almost every w £ Run(p(k), Z)) there exists m £ N such that the counter value is at 
least £ in every configuration w(m') where m! > to. Consequently, Fa(w ) = Fs for almost all w £ Run(p(k), Z). 

If Z = Ru(S), where S' is a BSCC of Z’a such that ts = 0, then the configurations of Z with zero counter are visited 
infinitely often by almost all runs of Run(p(k), Z), but the expected number of transitions between two consecutive visits 
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to such configurations is infinite (see 0)- In other words, visits to configurations with zero counter have zero frequency for 
almost all runs of Run(p(k ), Z). Consequently, Fa(w) = Fs for almost all w G Run(p(k), Z). 

Finally, consider the case when Z = R, where R is a type II region determined by p £ S where S' is a BSCC of Ft' a 
satisfying ts < 0. Let Vs be a finite-state Markov chain where the set of vertices is {qj \ j G {0,1}, q(j) G f?} and the 
transitions are defined as follows: 

• qo 3 *’ Tj in V s iff 17(0) A r(j) in Ma (where j G {0,1}); 

« qiA-ro in Vs iff x = [qfr] > 0. 

Note that the sum of the probabilities of all outgoing transitions of every vertex r± of Vs is equal to one, because almost all 
runs of RunM A {q{ 1)) visit a configuration with zero counter (see j5j). Also note that Vs is strongly connected. For every 
transition of the form q\ —> ro in Vs, we define the following conditional expectations: 

• E[L | qi —M"q], the conditional expected length of a path from q( 1) to r(0) in A4a, under the condition that g(l) reaches 
r(0) via a path where the counter stays positive in all configurations except for the last one; 

• E[fi= s | q\ —^ro], the conditional expected number of visits to configurations with control state s G S along a path from 
q{ 1) to r(0) in A4a (where the visit to r(0) does not count), under the condition that <7(1) reaches r(0) via a path where 
the counter stays positive in all configurations except for the last one. 

Let px> s be the invariant distribution of Vs- Then 

m= E AkD s (<?o) + E Vvs(qi) ■ x ■ E[L | q 1 -> r 0 ] 

qo'inDs r/i Ar„irrDs 

is the average number of transitions between two consecutive visits to configurations q[i), r(j) of R, where i + j < 1, in a 
run initiated in a configuration of R. Similarly, 

£[s] = E (qi)-x- E[# s I qi ->• r 0 ] 

qi—>r 0 inDs 


is the average number of visits to a configuration with control state s between two consecutive visits to configurations q{i), 
r(j) of R, where i + j < 1, in a run initiated in a configuration of R (the visit to r(j) does not count). Now we define a 
vector Fx> s : Pat a —> R as follows: 

. Fvsiqi 0)) = 0 for all q( 0) £ R, 

• ^x>s (<?(*)) = 0 f° r a b q(*) suc b ^at R does not contain any configuration matching q(*), 

• F-Ds ioi 0 )) = PD s (qo)/E[L\ for all g(0) G R, 

• Fxi s (q(*)) = E[q\/E[L] for all q(*) such that R contains a configuration matching q(*). 

By applying strong ergodic theorem (see, e.g., GO)), we obtain that Fa(w) = Fx> s for almost all w G Run(p(k), R). 

Since the transition probabilities of Vs may take irrational values, the numbers involved in the definition of Fx> s cannot be 
computed precisely. By Theorem 3.2 (B.b. 1) of 0, we have that both E[L \ q\ —>ro] and E[fi s \ qi —M"o] are bounded by 
a = 85000|(5| 6 /(a:^^ + ^' • tg), where x m i n is the least transition probability of Ma- Note that ||a| is polynomial in ||^4||. 
Using this bound, a simple error propagation analysis reveals that if the transition probabilities of Vs, all components of the 
invariant distribution /ru s , and the conditional expectations E[L \ q\— >ro], E[fi s \ q\— >-ro] are computed up to a relative 
error e/(42 • \Q \ 2 ■ a), then the relative error of every component in the approximated Fx> s is bounded by e. By f2l) , the 
transition probabilities of Vs can be approximated up to an arbitrarily small positive relative error in polynomial time. The 
values of the conditional expectations can be efficiently approximated by applying the results of |(5) (in 0, the results are 
formulated just for E[L \ q\— >ro], but their extension to E[fi= s \ q\— ►r’o] is trivial). The invariant distribution pxi s can be 
efficiently approximated by, e.g., applying the result of J8] (see also 0 for a more comprehensive overview) which says that 
if the transition matrix M-p s of Vs is approximated by M! Ds , then \\pv s — F-i) s ||| < Q ■ |||Mx> s — Mj, 

and 


invariant distribution of M! Ds 


where p! Vs is the 


1 

q = — • max 
2 j 


maxj^ rriij 


m-j 


Here rriij, i ^ j, is the mean first passage time from state i to state j, and m u is the mean return time to state j, where all 
of these values are considered for Mv s ■ Since the least transition probability of Vs is at least (see lfl2l ) and Vg has at 
most 21 Q | states, we have that m,; ? and mu are bounded by 2\Q\/xffif . This means that every component of px> s is bounded 
by /2|Q| from below, and p is bounded by IQI/a^a, from above. Hence, it suffices to approximate the transition 

probabilities of Vs up to the absolute error ea^in / ( 168 |< 5 | 5 «) and compute the invariant distribution for the approximated 
transition matrix M! Ds . ■ 
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Lemma 0 Let p{k) be a configuration of A. Then almost every run initiated in p(k) eventually stays in precisely one zone 
of A. Further, for every zone Z and every rational e > 0, there is a P £ Q computable in time polynomial in ||*4||, |e||, and 
k such that P approximates P(Run(p(k), Z)) up to the relative error e. 

Proof: First, observe that every region is a part of some zone, except for non-empty type IV regions determined by BSCCs 
of Z'a with negative trend. If R is such a type IV region and S the associated BSCC where Is < 0, then almost all runs 
initiated in a configuration of R visit a configuration with zero counter infinitely often. Consequently, almost all runs initiated 
in a configuration of R visit a type I region, which means that V(p{k),R) = 0. Thus, by applying Lemma [2] we obtain that 
almost every run initiated in p(k) eventually stays in precisely one zone of A. 

Let Z be a zone of A. We proceed by considering possible forms of Z. 

• Let Z = R, where R is a type I region determined by a control state q. Note that the problem whether R = 0 is decidable 
in time polynomial in ||.4.||. If R = 0, then V(Run(p(k ), Z)) = 0 and we are done. Otherwise, V{Run{p{k), Z)) = 
V(Run{p{k) —> *q( 0))). 

• Let Z = R where R is a type III region determined by a BSCC S of such that tg < 0. Then V(Run(p(k ), Z)) = 
V(Run(p(k ), t s)) (note that the equality holds even if R = 0). 

• Let Z = R h (S) U Rui(S) U f?/y(-S'), where S' is a BSCC of ^4 such that tg > 0. Then V(Run(p(k ) 1 Z)) = 

V(Run(p(k)As))- 

• Z = Rji(S), where S is a BSCC of Zfi such that tg = 0. If the type III region determined by S is non-empty (which 
can be checked in time polynomial in |„4||, then Ru(S) = 0 and we are done. Otherwise, every configuration of S' x N 
can reach a configuration with zero counter. Let 4CSxS be a binary relation such that s °4 t iff s(0) —>t( 0) in Ad a- 
Note that ^4 is computable in time polynomial in ||*4J. Hence, we can also efficiently compute the BSCCs of (S, °4), 
and determine all non-trivial BSCCs K of (S, °4) such that for some (and hence all) q £ K we have that post* (q(0)) is 
infinite. Each non-trivial BSCCs of (S, °4) corresponds to a type II region determined by a control state of S, and vice 
versa. Let us fix some control state qi( £ K for each non-trivial BSCC K, and let K, be the set of all non-trivial BSCCs 
of (5, ■H>). Then P (Run(p(k), Z)) = T /KeK: 'P(Run{p(k) ^*q K (0))). 

• Let Z = R, where R is a type II region determined by q £ S where S' is a BSCC of Zfi such that tg < 0. If q belongs 
to a non-trivial BSCC of (S, ^4) (see the previous item), then V(Run(p(k ), Z)) = V(Run(p(k) —> *q(0))). Otherwise, 
V(Run(p(k), Z)) = 0. 

Hence, it suffices to show how to efficiently approximate V(Run(p(k),fs )) an d V(Run(p(k) —>*q( 0))) where we may further 
assume that V(Run(r(f)—>*q( 0))) = 1 for every r(£) £ post*(q(0)). In the following we assume that k = 1 and we prove 
that these probabilities can be approximated up to a relative error e > 0 in time polynomial in ||_4.|| and ||e|| (for k > 1, we 
simply introduce k — 1 fresh control states that are used to increase the counter from 1 to AO- 

Let us fix q £ Q such that V(Run(r(£)—>*q( 0))) = 1 for every r(£) £ post*(q(0)). Hence, if p( 1) £ post*(q{0)) then 
V{Run(p{\) —> *q{0))) = 1, and if p{ 1) ^ pre*(q( 0)) then V(Run(p(l)—t*q(0))) = 0. Now assume p( 1) £ pre*(q( 0)) \ 
post*(q(0)). We construct a finite-state Markov chain £ q where the set of vertices consists of all r 3 where j £ {0,1} and 
r(j) £ pre*(q( 0)) \ post*(q(0)), and two fresh vertices good , bad. The outgoing transitions of a vertex r :) are determined as 
follows: 

• ro Sj iff Sj is a vertex of £ q and r-(0) Zy s(j) in 

• ro A good iff x > 0 is the total probability of all transitions r(0) -=4- s(j) in A4a suc h that s(j) £ post* (q(0)); 

• ro Zy bad iff x > 0 is the total probability of all transitions r(0) ^4 s(j) in JAa such that s(j) pre*{q(ff))\ 

• ri Zy s 0 iff so is a vertex of £ q and x = [r{s] > 0 

. n Zy good iff x = E s (o)e pos t*(9(o))[ r W > °! 

. n Z> bad iff X = [rt] + E s (0)^pre*(9(0))H S ] > °; 

• good -4 good, bad —> bad. 

It follows directly from the construction of £ q that V(Run(p(l) —> *q(0))) is equal to the probability of reaching good from 
Pi in £ q . Note that £ q is an absorbing finite-state Markov chain with two absorbing vertices good and bad. Let E q be the 
other (transient) vertices of £ q , and let U : E q -£ R be the unique vector such that U v is the probability of reaching good 
from v in £ q . Then U is the unique solution of the system x = Ax + C, where A is the \E q \ x \E q \ transition matrix for the 
transient part of £ q and C v is the probability of the transition v -£ good in £ q (if there is no such transition, then C v = 0). 
This system can be rewritten to the standard form (I — A)x = C. Note that (I — A )~ 1 (i.e., the fundamental matrix of £ q ) 
satisfies ||| (I — j 4) _1 ||| < max„ £ E q m v , where m v is the mean time of reaching an absorbing state from v. Since every vertex 

of E q can reach good in at most 2\Q\ transitions and the probability of each of these transitions is at least xj^ (see lfl2l ). 
we obtain that m v is bounded by 2|Q|/a; II [x i l . Since ||| I — H||| < 1, we obtain that the condition number of I — A, i.e., 
HI J — A||| • J|(7 — A) _1 ||| is bounded by 2|Q|/x^|^ from above. By applying the standard result of numerical analysis (see, 
e.g., S3U), we obtain that if the coefficients of A and C are approximated so that the resulting matrix A! and vector C satisfy 
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1A - A'|||/|||/ - A If < ex ^9j /8|Q| and ||| C - C'\/ ||C| < ex 2 J9j /8|Q|, then the unique solution U' of (I - A! )x = C' 
satisfies ||| U — [/ , |||/|||[/||| < e. Since such A' and C' are computable in time polynomial in ||A|| and ||e|| ETl . |5l , we are done. 

Now let S' be a BSCC of c €a- First, realize that if we change every rule (r, k, s) £ 7, where r belongs to a BSCC of 
^4 different from S, to (r, —l,s), then the resulting pVASS A! satisfies V(RunM A (p(l)As)) = 'P{R un M A r (p(l), t))- 
To simplify our notation, we directly assume that V (Run(p(l) A s )) = V(Run(jp( l),t)), and we show how to approximate 
V(Run(p( 1), T))- Let Diverge be the set of all configurations q(l) such that q £ Q and [gf] > 0. If p(l) ^ pre*(Diverge), 
we have that V(Run(p( 1), T)) = 0. Otherwise, we construct a finite-state Markov chain Q where the set of vertices consists 
of all rj where j £ {0, 1} and r(j) £ pre*(Diverge), and two fresh vertices good, bad. The transitions of Q are determined 
as follows: 

• ro -4- Sj iff Sj is a vertex of Q and r(0) -4 s(j) in AT. 4 ; 

• ro -4 bad iff x > 0 is the total probability of all transitions r(0) -4 s(j) in Ad a such that s(j) pre* (Diverge)’, 

• 77 -4 so iff so is a vertex of Q and x = [r\|.s] > 0; 

« 77 -4 good iff x = [ 7 't] > 0 ; 

. n 4 bad iff X = ^ s(0 )t P re*(Diver g e)Vls\ > 0l 

• good —> good, bad bad. 

It is easy to check that V(Run(p( l),t)) is equal to the probability of reaching good from p\ in Q. The rest of the argument 


MQr 


is the same as above, the only difference is that now we also employ the x 
probability of the form [rf], where t is the trend of S (see Theorem 4.8 in ||5l). 


t 3 /(7000 • |Q| 3 ) lower bound on positive 


Appendix B 
Proofs of SectionHyI 

The whole Appendix fBl is devoted to proofs of the following three theorems. 

Theorem 0 Let S be a BSCC of^A such that tg( 2) < 0. Moreover, let R be any type-II region of A 2 determined by some 
state of S. Then there are cii , bi £ K. >0 and Z\ £ (0,1) computable in polynomial space such that the following holds for all 
p £ S such that p( 0) £ R, all n £ N, and all i £ N + : 

(1 

<QO/\x\ 

Moreover, if £p( n ,o) < 00, then it holds 

In particular, neither of the bounds depends on n. 

Theorem [4] Let S be any BSCC of ( C’a such that t$( 2) < 0. Furthermore, let R be any type II region of A 2 determined by 
some state of S and satisfying tr > 0. Then there are numbers 02,^2 > 0 and 0 < zi < 1 computable in space bounded by 
a polynomial in ||A|| such that for all configurations p(n,0), where p( 0) belongs to R, the following holds: 

\p(n, 0) -A <?(0, *)] < a 2 ■ z%' b2 

Theorem |5] Let R be any type II region of A 2 such that tr < 0. Then there are numbers 03 ^ 3 , cfa > 0 and 0 < Z 3 < 1, 
computable in space bounded by a polynomial in ||A||, such that for all configurations p(n,0), where p( 0) belongs to R, and 
all q £ Q the following holds: 

\p(n, 0 )^* q( 0 ,*),i] < i.a 3 ’ 4 nTRb3+id3 

for all i > 3^ where H is a computable constant. 

We use the following additional notation: we denote by Run(pv —>•* qu,i) the set of all runs w £ Run(pv) such that 
T(w) > i, w(i) = qu, and for all 0 < j < i we have w(j) qu. Note that the probability of Run(pv —>* qu, i) is exactly the 
number [pv —>■* qu,i]. 


> i) < ai ■ z bl ' 1 
/ at • z b f 

~ 1 -a*’ 


A. Martingale Techniques 

In this subsection we use the techniques of martingale theory to prove several technical lemmas that are crucial for the 
analysis of those sets of configurations C[Ri,R. 2 \ such that R 2 is a type II region in which the corresponding counter has a 
tendency to decrease. 

To this end, fix a region R of A2 whose type is II and which is determined by some p £ S, where S satisfies tg( 2) < 0. 
We use the stochastic process {m^}£defined in o as follows: for every I £ N we put 

m \ x ^ ~ T R • ^ + g(x^) [p^l if > 0 for all 0 < j < l, 
m m (M) otherwise. " (1) 
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(Here g is a suitable function, defined precisely in 0, assigning numerical weights to configurations of A‘j-) In other words, 
the value m^ t> ( w) of the f-th configuration of a run w is obtained by adding the value of the first counter x[ f > (w) to the value 
of the second counter x^f 1 (w) weighted by the function g and by subtracting £ times the trend tr. 

In 0 we defined the function g in such a fay that the process {rn ( 'A)ff 0 satisfies several important properties: First of all, it 
is a martingale E2, which intuitively means that the expected value of the £-\h configuration is always equal to the observed 
value of the (l — l)-th configuration, even if we are given the knowledge of values of all configurations up to the (t — l)-th 
step. 

A second crucial observations proved in 0 is that grows more or less linearly with x£\ 

Lemma 20 (0). There is a number C > 0 computable in space bounded by a polynomial in ||.4|| such that for all p £ Q 
and n > 1 it holds |fl(0)[p]| < C and |< 7 (ro)[p]| < C ■ n. 

However, to apply powerful tools of martingale theory, such as the Azuma’s inequality, we need to show that {m^}£L 0 
has bounded-differences. This is covered in the following lemma, whose proof combines several facts shown in 0 with 
rather involved techniques from the theory of stochastic matrices. For better readability, we prove this lemma separately in 
Appendix IB-HI 

Lemma 21. There is a bound B > 1 computable in in space bounded by a polynomial in ||.4|| such that \— rnA ' 1 | < B 
for every £ £ N. 

The power of martingale techniques is illustrated in the following lemma, which will be handy in the proof of Theorem 0 

Lemma 22. Let R be a type II region of A 2 determined by some p £ S, where S is a BSCC of such that ts(2) < 0. 
Moreover, let p(n, 0) be any configuration such that p(0) £ R. Denote by Repeat the set of all runs w such that w(0) = w(i) 
for some positive i such that 4 B 2 /t^ < i < T(w). Then 

'Pp(np') (Repeat) < 

Proof: For any run w £ Repeat let TRepeat(w) be the smallest i > AB 2 /tr such that w(i) = w( 0) (for such w we have 
TRepeat(w) < T(w)). We have 

OO 

Rp(n,o) (Repeat) = E V p (n,o)(Repeat AT Repeat = i) (2) 

M4BV41 


Now any run w initiated in p(n, 0) satisfies m^°\w) = 
satisfies TRepeat(w) = i satisfies m^(w) = n + g(0)[p] 

Rp{n,t)) (Repeat A TRepeat = i) 

From Azuma’s inequality we get 

P p (n,o)(\m M -ro (0) | >i- |tr|) < 

Combining this with 0) and 0 we get 


n + g(0)\p). Similarly, any run w £ Run(p(n, 0)) (T Repeat that 
- i ■ tr. Hence, 

< Pp(n,o)(h W - to(0) | > i ■ |tr|). (3) 


2 exp 


~ t r ■ 1 

2 iB 2 


= 2 exp 


~ T n : 1 

2 B 2 


Rpin.o) (Repeat) < 
< 



~ t r ' 

2 B 2 ) 

1 

2 ' 


< 


exp (T^l • • (1 - exp ( - r|/2H 2 )) 


The following lemma, which is crucial for the proof of Theorems Q] and [5] is also proved using Azuma’s inequality. 

Lemma 23. Let R be a type II region of A 2 determined by some p £ S, where S is a BSCC of ^4 such that ts( 2) < 0. 
Then there are a',b',b" £ R>o and A £ (0,1) such that for all p,q £ Q, where p(0) £ R, and all i,n,n' £ N satisfying 
i > (2 • (C + 1) • ti')/\tr\ the following holds: If either tr > 0, or —tr ■ | > n, then 

[p(n, 0) g(0, «'),*] < a' ■ ( c , ) n ' ri? ' b ' +i ' b " 


Moreover, a',b',b",c' are effectively computable in polynomial space. 
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Proof: Denote v = (n, 0) and u = (0 ,n'). For better readability, we denote by r the number tr, where R is the region 
of A 2 containing p(0). Let w £ Run(pv —)•* qu,i). Then 

(■ m ^ — m^)(w) = Xi\w) — t • i + g[x2\w))[p^\w)] 

— x^\w) + t ■ 0 — g(x2°\w)) [p® (tt;)] 

= -ri + g(ri)[q\ - n - g{ 0) \p] 

= 9 {ri)[q] - g(0)[p] - n-ri. 


Thus, 


pv —>* qu, i] < V(m ^ — m ^ = g(n')[q] — p(0)[p] — n — ri). 


Note that if i < n, then \pv — >* qu, i] = 0. Assume i > n. Since i > (2 • (C + 1) • ti')/\t\ > (2 • \g(n')[q] - p(0)[p]|)/|r|, we 
have 


|p(n')[g]-5(0)[p]|<|r|-. 

Thus, denoting Z = g(n')[q] — p(0)[p], the following holds: 

« If r < 0, then from © we have Z > r| and thus Z — t| > 0. Hence, 


(4) 


[pv —>* qu, i] < V(m ^ — to® > —n — r—). 


(5) 


• If r > 0, then from © we have Z < |r|| = r|, and thus Z — r| < 0. Hence, 

[pu —qu, i] < P(m^ — to® < — n — r^). 

Now we apply the Azuma’s inequality. First consider r > 0. Then, by Azuma’s inequality, for all i it holds 


n 


to ' — to 


(o) 


0 . 

< —n — t— ) < 2 exp 


= 2 exp 


— (n + §i) 2 


= 2eXP '2H 


2-H-i 
'-n 2 -rm- 

2-B-i J 

1 ( — n 2 t 2 

- nr — *— 

1 4 

^2 


- 2exp ( ‘h 1 ~ nT ~ i ~ 


< 2 exp ((*+ 


(For the last inequality we used the fact that r 2 /4 < 1.) Combining this inequality with © we obtain 


[pv — qu, i] < 2 exp 


8H 


(* + nr) 


( 6 ) 


It is now easy to compute a', b', b", d from the statement of the lemma: it suffices to put a' = 2, =1/2 and b' = b" = 

where x is a number, computable in polynomial space, such that r > x > 0 (x can be computed in polynomial space since r 
can be encoded in Tarski’s algebra). 

Now consider r < 0. To apply Azuma’s inequality in this case, we need to assume that — r| > n. Then, as above. 


V(m^ — to*- 0 -* > — n — t —) < 2 exp 


1 

2 B 


and combining this with © yields 


[pv ->* qu. i] < 2 exp ( — ( —nr — i— 


Numbers a!, b', b", d can be now easily computed (we put b' = 1/2H and b" = x 2 /8B, where x is as above). 
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B. Proof of Theorem [7] 

Fix a region R of A 2 that satisfies the assumptions of Theorem 0 

For the purpose of this proof we define the value T(w) also for finite paths w: we put T(w) = inf {* | 0 < z < length{w ) A 
x'f ' 1 = 0}. Given a finite path or a run w, we denote by LVisit(w ) the largest number k such that x^\w) = 0 and for all 
0 < i < k we have x^\w) > 0 (if there are infinitely many such k, which is possible only if length{w) = T(w) = 00 , we 
put LVisit(w) = 00 ). Further, for w such that LVisit(w) € N we let Last(w) be the configuration w{LVisit{w)). Finally, for 
a finite path or a run w and a configuration qu we denote by # qu (w) the number of occurrences of qu on w before zeroing 
the first counter. Formally, we put 

#qu{w) = |{* G N I 0 < i < T(w) A w(i) = qu}\. 

We have 

OO 

'Pp{n,o)(T < OO A x'P >i) = EE P p( nM T < 00 ^ * A Last = q{k 1 0)) 

q&Qk= 1 


OO OO 

= EEE r P p (nfi)(T < OO A x^ > i A Last = q(k , 0) A # q (k,o) = rn) 

q€zQ k= 1 m= 1 


(7) 


Now denote by the event T < 00 A x^ >i A Last = q(k, 0) A # q (k,o) = m - It holds 


A 


i,m 

q,k 


u 


u 


Run(w ■ w 1 ). 


w£FPath(p(n,0 )) 

i£q(k,0) = LYl 

w (length(w))—q(k, 0) 


w' £ Run(q(k ,0)) 
T(w')< oo 
x 2 T ^ (w')>i 
Aji” ,)_1 x!f\w')>o 


(Here w ■ w' = w( 0), w(l), ..., w{length{w) — 1), w/(0), _) It follows that 

'Pp(n,0)i A q!k) = E 'Pp(n,0)( Run ( W )) • ( E 7Vn,0)(-Rwi(u/))) 

w£FPath(p(n, 0)) w' €Run(q(k, 0)) 

q(k,0) = Ln T(lt/)<00 

w(length(w))=q(k,0) x^\w')>i 

AjiT' , - 1 ^ ) K)>o 

T-l 

= 'Pp(n,0){#q(k,0) > ™) ■ V q (k,0){T < OO A 4 T) > z A /\ 4 J) > °)- ( 8 ) 

J=1 

Note that every run w initiated in q(k,0) that satisfies x;, ; ('tf) > z and f\j =1 x)f (w) > 0 must have a prefix of length at 
least z + k such that for every 0 < j < i + k we have 4^ (tu) > 0. It follows that 

T -1 

'Pq(k,0)( T < OO A 4 T) > 1 A 4 J) > 0) < 7^,(0)(L > i + &)• (9) 

i=i 

Here V q (pAL > i + k) is measured in A 2 and L assigns to a given run w of A 2 either the least k > 0 such that the counter is 
zero in fc-th step of w, or 00 if there is no such k (intuitively, L(w) is the number of steps in which w (re)visits a configuration 
with zero counter value for the first time). 

By (5] Section 3.1], we can compute, in polynomial time, a number z such that 

2 d i+k 

'P q (Q){L > i + k) < 1 + 2\z\/\t 2 (s)\ + |Q| • ^ ^ , 

where d = exp ( — 8 ( ~4*4(2U+i) a ) ^ (0,1). Using this knowledge, we can easily compute, in polynomial time, numbers 
a, b £ R>o, and c € (0,1) such that 

V q{0) (L>i + k ) < a-c b < i+k l 

Plugging this into (0, ([8]) and (O we get 

OO OO 

P p(71i0) (T<ooa 4 t) >i) < J2J2 a ' cHl+k) ■ (E ^. ( n,0)(#,(fc,0) >rn)). (10) 

qEQ k— 1 m=1 
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We now turn our attention to bounding V p ( n fi) Of q (k,o) -2 m )■ From Lemma l22l it follows that for any h G N it holds 

'Pp(n,o)(#q( i k,o) > ^B 2 /tr) < 2~ h . 

From this it follows that 

00 4 o2 00 on2 

'Pp(n,o)(#q(k,o) > m) < — • 53 2 = “re¬ 


plugging this bound into (ITOt we get 

OO 

£p(n,o) < 00 a x'p > t) < y y< 


6-(i+fe) 


h—0 


8 B 2 


a ■ c b ■ 8 B 2 

(1 


J)-i 


q eQk =1 '-R 

and from this form the numbers 01,61, Ci in the statement of Theorem [2 can be easily computed. 
Finally, when V p ( n p)(T < 00) = 1, we have 


E p(n,o) (4 T) ) = y, V p{n, o)(4 T) >*) < 5Z ai ‘ c i 11 


ai • c/ 


i> 1 


i> 1 


i-cr 

C. Proof of Theorem [7] 

We start with a simple corollary of the facts that we proved so far. The corollary will be also useful in the proof of Theorem[5] 


Corollary 2. Let R be a type II region of A 2 determined by some p £ S, where S is a BSCC of Af such that ts{ 2) < 0. 
Moreover, let p(n , 0) be any configuration of A such that p(0) £ R. Denote k = 2{c +\) ’ w ^ ere E as in Lemma \20\ Then 
for arbitrary such that either i > 2 • ti/\tr\ or tr < 0 it holds 

[p(n, 0) q{ 0, *), i] < i ■ k ■ a 1 ■ (c^-rn-b'+i-k-b" + 

i-ci 1 

where a' ,b r ,b" ,d are as in Lemma \23\ and a±,bi,ci are as in Theorem\3\ 

Proof: We have 

i \i-k\ i 

b( n -°) ->■* 9(0,*),i] = y bKO) -»* q(0,n'),i] = y \p(n, 0) ->* q(0, n'), i\ + y \p(n, 0) ->* q{Q,ri), i] 

n '=0 n '=1 n' = [i-k+l\ 

i i 

< i ■ k ■ a' ■ (c') n - TR - b ' +i - b " + y [p(n,0)->*g(0,n / ),»] < i ■ k ■ a'■ (c') n ' TR ' b ' +i ' k ' b " + y 

n' = \_i-k- l-lj n'=[i-k-\-l] 

< i-k-a’ ■ (c'^-ru-b'+i-k-b" + . J-bi-k' 

1 -c b f 

Here the first inequality on the second line follows from Lemma l23l (note that any n' < i ■ k satisfies i > (2 (C + l)n')/\ tr\, 
so the assumptions of this Lemma are satisfied), while the second inequality follows from Theorem [3] ■ 

We now proceed with the proof of Theorem [4] Since tr > 0, from Corollary [2] we get 


61 •i 

a 1 • cy 


\p(n, 0) —>* q( 0, *), i\ < i ■ k ■ a' ■ ( c') l ' k ' b + 


a 1 


A-bi-k 


1 61 1 

1 - Cl 


But then 


(-XJ 

[p(n, 0) ->* t?(0, *)] = y\p(n, 0) ->* q( 0, *), i] < 


n - k ■ a' 


{c') 


J\n-k-b" 


a 1 


n-bi-k 


( l -( c ') fc ' 6 ") 2 V °' 1 

The numbers 02, &2, ~2 in Theorem |4] can be straightforwardly computed from this bound. 


D. Proof of Theorem \5\for ts (2) < 0 

In this subsection we prove Theorem [5] under the assumption that the region R is determined by some p £ S, where S is a 
BSCC of c toA with ts( 2) < 0. The case when is(2) > 0 is handled separately in the next subsection. 

We put 0,3 = 2 • max{fc • a',ai/(l — cj 1 )}, 63 = b' , Z 3 = maxjc'jCi} and 63 = k ■ min{h",6i}. From Corollary [2] it 
immediately follows that 

[p{n, 0) -►* 9(0, *), i] < i ■ 03 ■ (z 3 r TRrb3+ ^ 3 

whenever i > -f 11 —. From this, the numbers a 3 , 63, z 3 in the statement of the theorem can be easily. 
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E. Proof of Theorem\5\for tg(2) > 0 

Intuitively, if the second counter starts high enough and tg( 2) > 0, then the probability of reaching zero in the second 
counter is negligibly small, and hence we may basically ignore the value of the second counter. We obtain a one counter 
pVASS, preserving the behaviour of the first counter, on which we may easily bound time to zeroing this counter using the 
previous results. 

Within this subsection we often operate with several probability measures within a single expression. To differentiate between 
them, we denote by V B the probability measure associated to a pVASS B. 

In the proof we use some known results on one counter pVASS. Let B be a one counter pVASS and let S' be a BSCC of 
such that tg f 0. As shown in the proof of Lemma 5.6 in f5] (see also Proposition 7 in (4)) one can compute, in time 
polynomial in the size of B , a bound hg £ N and numbers ag > 0 ,cg£ (0,1) such that for all configurations p(k), where 
p £ S, all states q of B, and all i > hg ■ k it holds 

'Pp(k) (Run(p(k) ->* q( 0 ),i )) < a s ■ Cg\ (11) 

Another crucial tool from the world of one-counter pVASS is the divergence gap theorem J5] Theorem 4.8], which bounds 
a positive non-termination probability away from zero. The theorem states that for any BSCC S of B with tg > 0 there is a 
number 5g £ (0,1) computable in polynomial time such that for every p £ S for which the probability of avoiding zero when 
starting in p( 1) is positive it holds 

J 2 'Pp i i ) (Run(p(i)^* q( 0 ))) < 1 -Ss- 

q&Q 

In m we proved that for any S with tg > 0 there is £ £ N such that for every p £ S the probability of avoiding zero counter 
from p(t) is positive. Now if R is in a type II region of B determined by some p £ S, then from every configuration q £ S 
such that (/(IS 1 !) £ R there is a zero-avoiding path from gd^l) to some r(£), so the probability of avoiding zero from <j , (|S'|) 
is also positive. From this it follows that for any p{k) £ R, where k > |S|, it holds 

J2'P^ k) (Run(p(k) * 9 (0))) < (1 - Sg) k ~ 

q&Q 

Both of the above results on one-counter pVASS were originally proved using a suitable martingale {rh^}e=ooo- We will 
need this martingale in this subsection as well, to prove some additional auxiliary results. The process can be seen 

as the one-counter analogue of the (substantially more complex) two-counter martingale which was defined in one 

of the previous sections. Formally, let us fix a one-counter pVASS B and a BSCC S of We define a stochastic process 
on runs of B by putting, for every (eN, 

-(«)_/ x i ] -ts-t + z(P {e) ) if x[ j) > 0 for all 0 < j < t, 

1 ro^ _1 i otherwise, 

where z is a suitable function assigning numerical weights to states in S. It was shown in Q that one can compute, in time 
polynomial in size of B, a function z such that 

• max Pj9eS | z(p) - z(q)\ < hg, and 

• the stochastic process {m^}^i 0 is a martingale whenever the initial state belongs to S. 

Now fix a two counter pVASS A and let a, c, and h be the maximal ag and eg and hg, respectively, among all BSCCs S 
of Ai and A>- Note that a, c, and h can be computed in time polynomial in the size of A. 

Lemma 24. Let R be a region of A 2 satisfying the assumptions of Theorem\5\whose corresponding BSCC S satisfies tg( 2) > 0. 
Then there are numbers 04 , 64 > 0, C4 £ (0,1) computable in polynomial time such that for every p £ S with p( 0) £ R, every 
n £ N, every i > max{2/m, 4| and every k > i /8 it holds 

\p(n, k ) -y* 9(0, *), i/2] < « 4 • (c/t) 1 ' 64 


Proof: Note that 

i /2 

[p(n, k) —>* 9(0, *), i/2] < P^(to(p(n) ^ 9(0),*/2)) + EE ? pw( to W fc ) r (0),j)). (12) 

, ~eQ j—0 

This is because the set of runs initiated in p(n, k ) that visits a configuration with zero value in the second counter before 
reaching zero in the first counter has the same probability as the corresponding set of those runs of A\ initiated in p{n) whose 
accumulated payoff (i.e. the change of the second counter which is encoded in labels of A\) does not drop below — k before 
a configuration with a zero counter is reached. We will bound both summands in (fl2l) separately. 


23 


( 13 ) 


For the first summand, the discussion at the beginning of this subsection shows that 

V p{n) {Run(p(n) ->* q{ 0), i/2)) < a ■ <a-cA. 

For the second summand we use the divergence gap theorem, see above. We have 

i/2 

< J2 V p(l)( Run (P( k ^* r ( 0))) ^ (l-^) fc - |Si < (1 -6)i. (14) 

rGQ j =0 rGQ 

(The last two inequalities follow from our lower bounds on i and k.) Combining ( 1 1 41 . ( |T3l >. and ( 1 1 21 we can easily compute 
the numbers 04 , 64 , C 4 from the statement of the lemma. ■ 

Now starting in p(n, 0) and assuming that £5(2) > 0, we may easily show that the second counter quickly grows with high 
probability. 

Lemma 25. Let R be a region of A 2 satisfying the assumptions of Theorem\5\whose corresponding BSCC S satisfies ts( 2) > 0. 
Then there are numbers as, b 5 > 0, C5 £ (0,1) computable in polynomial time such that for every p £ S with p(0) £ R, every 
n £ N, every i > 16 • ( ts ^ 2 ) ) 2, an> ^ ever y k 5= 8 ts( n ) ^ bolds 

Klnfi){ T k = i/2 A Over < i/4) < a 5 • cf b \ 

Proof: 

For a run w £ Run(p(n, 0) —»•* r(*. k )) we denote by 

• T k (w) the smallest £ such that x^\w) = k, 

• Over(w ) the largest £ < T k (w) such that x^\w) = 1, 

• COver{w ) the configuration w(Over(w)), 

• #q{m) the cardinality of the set {£ £ N | £ < T k (w) A w{t) = q(m)}. 

We have 


\p{n,0) —>■* r(*,k),i/2] < V$ nfi) (T k = i/2 A Over < i/4) + Vfi nfi) (T k = i/2 A Over > i/4). (15) 

To prove the lemma it suffices to give tail bounds on both summands in (IT5l >. 

Let us start with the second summand. Denote by A the set of all runs w £ Run(p(n, 0) —»•* r(*, k)) such that 

. T k (w) = i/2 

• there are at most y/i/A indexes i < T k (w) such that the second counter is equal to 0 in w(£). 

• w contains a sub-path w'( 0 ),..., w'(f) with the following properties: 

- i /2 > e> y/TJl-i, 

- the second counter is equal to 1 in w'( 0 ) and w'{i), 

- the second counter is positive in all configurations of w' 

Note that the last item in the definition of A is implied by the previous items. Next, denote by B the set of all runs w £ 
Run(p(n, 0) —>* r(*,k )) such that there are at least y/i/4 indexes £ < T k (w) such that the second counter is equal to 0 in 
w(£). Then 

KnP){T k = i/2 A Over > i/4) < V$ n> 0 ) (A) + V$ n> 0 ) {B). 

Again, it suffices to give tail bounds for both summands on the right-side of the previous equation. 

First we focus on the probability of A. Let w be any run initiated in some configuration q(l) of A 2 , here q £ S, such that the 
least £ for which w{£) has a zero counter satisfies £ > (\/i/2) — 1. Then rh^(w) — m (°) (w) < —£ ■ ts( 2) + max Pi9£ s | z(p) — 
z(q )| < — £ ■ ts(2) + h < ■ ts( 2), the last inequality following from our assumption that i > 16 • itjprj) 2 - Using this fact 
and Azuma’s inequality we get that 


V: 


p(n, 0) 


(Al)< 



8((fs(2)) 2 + h + l) 


< (a) ■ (o') 


j \ vA-A 


for suitable polynomially computable numbers // > 0, c' £ (0. 1). 

Now we turn our attention to B. Since R is a type II region, from every configuration q( 0) of A 2 that is reachable from 
p(0) there is a finite path of length at most \Q \ 2 ending in a configuration of the form £(|S'|), from which the probability of 
zeroing the counter is at most 1 — 6 , as argued above. Hence, denoting p ln j n the minimal non-zero transition probability in 
f < 4 2 we get 


V: 


p(n, 0) 


(A) < ((1 - p ^;) • (1 - 6 )) < a" • (c") 




:Vi-b" 
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for suitable poly normally computable numbers a", b" > 0, d' G (0,1). 

Now we go back to and bound the number V/ n 0 j (7/ = i/2 A Over < i/ 4 ). We can easily show, using similar 
arguments as in the proof of Theorem [3] that 

n-\-i/2 i/2 i/4 

E EE Ac, 0 )( 2 fc = — A a^ 2 '* = k A Over = o A COver = q(j, 1 ) A = m) 

q&Q j=n—i/2m—l o—O 

i/2 n+i/2 i/4 . o 

< > m ) ' (E 51 E ( T fc = A - o A 4 5_0) = kA[\ 4^ > 0)) 

m=l qeQ j=n—i/2 o=0 i=\ 

n+i/2 i/4 . ^ — o 

< |E E E ( T k = ^~oA 4 5_0) = kA f\ x 2 ] >°)- 

<jGQ j=n—i/2 0=0 ^__r=l_^ 

denote by X 

Any run w G X satisfies m^ _ °) — fft/ 0 ) < k — (| — o) • fg(2) + ft < & — (j — o) ■ £s(2). For k < s-tl( 2 ) ^is number is 
< (| — o) ■ ts{ 2), and hence we can use the Azuma’s inequality to get 

n+i/2 i/4 

nt,o)(^)<5E E E 2 ( c '") (i -° )6 '" 

q£Q j—n—i/2 o=0 

for suitable polynomially computable numbers ft'" > 0, c'" G (0,1). Hence, 

n+i/2 

'PfinfiM) <^E E 2(c"') (i/4) - 6 "7(l - ++'")) < a"" • (c""f 6 "" 

gGQ j=n—i/2 

For suitable polynomially computable numbers a"",b"" > 0, c"" G (0,1). The numbers as, 65 , C 5 can now be computed in 
polynomial time using the tail bounds given within this proof. ■ 

Now we finish the proof of Theorem^ We have Let H = max{2ft, 4 | , 16( tg ^ 2 ) ) 2 }- Then for alii > H ■ n it holds 


n+i/2 

[p(n, 0) -s>* g(0,*),i] = E E E r + + */ 2 H r + k ) ->* 9(0,*), i/2] 

fc<i rGQ i=n—i/2 

and thus, by Lemma l25l 

i/8 _ i/2 n+i/2 

[p(«,0) ->* <?(0,*),i] < 5^«5 -c^+ E E E b( n >0) -*■* r(£,k),i/2][r(£,k) -+* q( 0 ,*), i/2] 

k= 0 k=i/ 8 rGQ i=n—i/2 

But by Lemma [24] for k > i/8 we have that 

[r(£, fc) -)•* g(0, *),i/2] < 04 • (c4) l ' fc4 

This together with the previous equation gives us 


[p(n, 0 ) -s>* q{ 0 , *), i] < l - ■ C 5 5 + a 4 • (c 4 ) l ' &4 . 

The numbers 03, 63, Z3 can now be straightforwardly computed using the above inequality. 


F. Proof of Lemma \K)\ 

t>l 

Let C be the set of all configurations of the form q(0,m) G Clf+ii^] satisfying m < 01 ' C L where ai,&i,ci come from 

1—c 1 1 

Theorem [3 As explained in Section | lv ] Lemma [10] is an immediate consequence of the following 

Proposition 2 (The Attractor). Consider C[i?i,f? 2 ] where both R\ and R 2 are of type II. Assume that fg(2) < 0, tg( 1) 7 ^ 0, 
T Ri < 0 and tr 2 < 0. Then C[R±, R 2 ] is eagerly attracted to C. 

Given a configuration pv and a set A of configurations we denote by [pv -+* A, > i] the probability that a run that starting 
in pv visits A in at least i steps and does not visit A between the first and the last step. 
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Given a run w, we denote by T\(w), T^u;), and Ti^iw) the least k such that w(k) £ Cs[c\ = 0], w(k) £ Cs[c 2 = 0], and 
w(k) £ Zs = Cs[ci = 0] U Cs\c 2 = 0], respectively. 

The following lemma reformulates results of Theorem [3] Theorem 01 and Theorem [5] in a bit weaker but more transparent 
way. 

Lemma 26. Let us fix pv £ C[R\, R^]. There are effectively computable numbers a > 0 and 0 < b < 1 (depending on pv) 
such that the following holds: 


1) For all £ £ N we have 



[pv -r* Z s , >£] < 

a ■ b l 

2) 

For all £ £ N we have 



?w4 Tl2) + 4 Tl2) > t) 

-O 

VI 

3) 

For all q £ Q, met) and £ > m we have 



[q{m, 0) -v* Cs[ci = 0], > £} 

< a-b Vri ™ 


and 



[9(0, 771 ) -V* C S [C2 = 0], > £] 

< a-b 

4) 

For all q £ Q, m £ N and £ £ N we have 



'Pq(m,0){xf l) >£) < 

a ■ b e 

5) 

For all q £ Q, m£ffwe have the following: 



a) If ts{ 1) < 0, then for all £ £ N we have 


'Pq(0,m){x^' 2) > l) < a-b e 

b) If ts{ 1) > 0, then for all £ > m we have 

V q(0 ,m)(x^ 2) >£) < a-b^ 1 

Let w be a run starting in pv. Denote by Qq(vj) the least £ such that w(£) £ Cs[c 2 = 0], Given k > 1, denote by Q k (w) 
the least £ > O k -±(w) such that the following holds 

• If k is odd, then w(£) £ C\s [c.\ = 0]. 

• If k is even, then w{£) £ Cs[c 2 = 0]. 

Lemma 27. There are effectively computable numbers a > 0 and 0 < b < 1 such that for all k > 0 and all £ £ N we have 

V pv (Q k -Q k -i>£) < a- {b )^ 1 


Proof: We distinguish two cases k = 0 and k > 0. 

Case 0 O : Note that 

Vpvi&o >£) < [pv ->* Z s , > £/2] + V^x f 12) + xf 12) > 1/4) 

e/4 

+ ^2 ^P«(4 Tl) A p (Tl) = q) ■ [q(m, 0) -V* C s [c 2 = 0], > £/2] 

q&Q m=l 

e/4 

< a ■ b^ 2 + a ■ 6 //4 + J2J2 V P»i x ^ x) = mA P (Tl) = q)-a- bW 2 ~ m 

qeQ m—1 

< 3 • a ■ b'fti 1 


which can easily be rewritten to the desired form. 

Case 0 k : If either /,5(1) < 0, or k is odd, then using Lemma [27] 2., 4., 5. a), and induction one may easily prove that 

Vp V {x+ x^ k ^ > £) < a - b e 

(Intuitively, whenever we start in q(m, 0), then we reach Cs[c 1 = 0] with probability one and by Lemmal27l4.. the probability 
that the height of the second counter at the time is at least £ is bounded by a ■ b e (independently of m). The same holds for 
configurations <7(0,771) since t${ 1) < 0.) 
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Now assume that fs(l) > 0 and that k > 2 is even. Note that 

e/2 

Ppv(x { ® k) >£) < > i/2 ) + Y, J2 V ^ x 2 0t_l) = mAp (e ‘- l) = g) • Prfo.mjfrf 50 > 

q&Q m=l 

e/2 

< a ■ ^ ^ V pv {= to A p*^ 04 - 1 ) = q) ■ a - b^ e ~ m 

q€Q m= 1 

< 2 • a • 

< a' • (&')^ 

for suitable a' > 0 and 0 < 6 ' < 1 that are effectively computable and satisfy a ■ b^ < a' ■ (6') . 

So for arbitrary tg( 1) 7 ^ 0 and k > 1 we have that 


Vpvix^ + xf k) > l) < a! ■ [b') 


' . it!\^ 


Now using the same argument as for 0o, we obtain for k odd, 

e/2 

Vpv(O k - 0 fc _! >£) < 7V(4 0t-l) > e/2) + J2J2 ^(4 0t_l) = m A p (0 ‘- l} = q) • [q(m, 0) C s [c 2 = 0], > i] 

q&Q m—l 

e/2 

< a' • ( b'Y 12 + J2J2 ^»(*2 e ‘“ l) = m A P (0t_l) = 9 ) • «' • 

<?GQ m—l 

< 2 • a' • (&') v ^ 


which can easily be rewritten into the desired form. For k even the argument is similar with counters switched. ■ 

Now let us finish the proof of Proposition |2] Let w be a run starting in pv. Denote by Rounds (w) the least number k > 0 
such that w(T k (w )) € C. Denote by -1 Terrrik the set of all runs w starting in pv satisfying w(Q k (w)) ^ C. We write ->Term <k 
to denote the set of all runs w starting in pv satisfying w(Qj(w)) C for all 0 < j < k. Finally, we denote by -1 Term the 
set of all runs that do not visit C before visiting a configuration of Cs[c\ = 0] \ C. 

It follows from Theorem [3] that there is an effectively computable constant 0 < c < 1 (independent of v{ \)) such that for 
all q e Q and all to € N we have 

'Pq(m,o)(-'Terrn) < c 

For all k > 1 odd holds 

P pv (^Term k A^Term <k ) = ^^2 'P pv {xf >k ~ 1 ^ =m Ap^ 6 ‘ -1 ' = q A ~^Term k A -*Term <k ) 

qeQ ragN 

= ^ ^ P P v(x ( f k ~ 1 ^ = to A p( 0fc -^ = q A -iTerm<fe) • Vpv(-<Term k | x^ >k ~ 1 ^ = to A p^ e>k ~ 1 '> = qf\ Term <k ) 
qGQ m£N 

= ^2^2 'Ppvix i e ' s_l) = m Ap (6k - l) = q A^Term <k ) •'P g ( m ,o )(->Term) 

qGQ m£N 

< ^2 ^2 ^ ? p«( a 'i 0fc_1 ^ = tn A p( 0fc -^ = q A Term <k ) ■ c 
qeQ me N 

= V po (-‘Term <k ) ■ c 

For all fc > 1 even holds 

Vp V (^Termk AmTerm <k ) < Vpu^Term^) 

Now since -iTerm< k = ~iTerm k -1 (T ->Term < k -1 we obtain, by induction, that Vpv{Rounds > v/fc) < (c)(L^J)/ 2-1 . Thus 
for all i > N we have 

1%/iJ 

[pv —7* C, > i] < V(Tours > [v^J) + ^ 7^(0^ — 0fc-i > L^J) 

k =1 

<( g )(LV7j)/2-i + y-. 6(S) L^j 

which proves that pv is eagerly attracted to C. 
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G. Geometric Sums of Stochastic Matrices 

For the proof of Lemma |2T] in Appendix IB-HI we will need a general lemma on geometric sums of stochastic matrices, see 
Lemma 1291 below. For the proof of Lemma [29l we use a coupling argument on finite-state Markov chains. As a preparation we 
first prove the following lemma on finite-state Markov chains. 

Lemma 28. Consider a finite-state Markov chain on a set Q of states with |<5| = n. Let y m in denote the smallest nonzero 
transition probability in the chain. Let p £ Q be any state and S C Q any subset of Q. Define the random variable T on runs 
starting in p by 


k 

undefined 


if the run hits a state in S for the first time after exactly k steps 
if the run never hits a state in S . 


We have V(T > k ) < 2 c k for all k > n, where c := exp(—y^ in /n). Moreover, if V(T < oo) = 1, then we have 
ST < hn/y ^ in , we we write S for the expectation with respect to V. 

Proof: If y m in = 1 then all states that are visited are visited after at most n — 1 steps and hence V(T > n) = 0. 
Assume y m i n < 1 in the following. Since for each state the sum of the probabilities of the outgoing edges is 1, we must have 
ymin < 1/2. Call crash the event of, within the first n — 1 steps, either hitting S or some state r £ Q from which S is not 
reachable. The probability of a crash is at least > y/ rm , regardless of the starting state. Let k > n. For the event where 
T > k, a crash has to be avoided at least I 11 times; i.e., 

V{T>k)<(l-y n min )^ . 

As I I > — 1 > - — 1, we have 

L n— 1 J — n— 1 — n 


V[T >k)< 


1 - y r ‘ 


(^-ymi n ) 1/n ) k <2-((l-t/”iJ 1/ ")' 


= 2 • exp - log(l - yl 
. n 


< 2 • exp ( - • (~y" in ) ) = 2 ■ c . 


Moreover, if V(T < oo) = 1, we have: 

OO 

ST = ^2 V(T > k ) 

k= 1 

oo 

< n + ^ 2 c k 

k -o 

2 

= n H--- - —r- 

1 - exp (-y^iJn) 

4 n 

< n H- 

Vmin 

< ^n/Vmin 


as exp(-y” m/ n ) < 1 


ymin 

2 n 


Now we are ready to prove the following lemma. 

Lemma 29. Let G £ [0, l]^ x< ^ be a stochastic matrix with only one BSCC. Let y m i n denote the smallest nonzero entry 
of G. Let r £ be a vector. Define f(n) := '^ 1=0 G 1 t for all n £ N. For any vector v £ let us define := 

max pi p 2g Q \v\px\ - v[p 2 \\- Then we have \f(n)\ diff < C \r\ diff for all n£N, where C := lO\Q\/y^j n . 

Proof: Let n £ K., and define r + := r + k1. Note that 1^+1^ = \ r \diff- Since G is stochastic, we have for all n £ N: 


n— 1 

E GV + 

= 

n— 1 

ntv 1 + G l r 

2—0 

diff 

2—0 


= |ntcl -I- /(n)| 


diff 


So in the following we can assume without loss of generality that r > 0 and r[p\ 


diiff ~ \f( n )\diff 

= 0 for some p £ Q, so that we have: 


I diff 


= PI 


(16) 
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Consider the finite-state Markov chain on Q induced by G. Let pi, p 2 £ Q be arbitrary states. We define random runs 
A-[°\ A-j 1 ^,... and 4 0) , a['\... in the Markov chain, with x \' :> , A 2 ^ £ Q for all i€N, and A-(°' 1 = p\ and X^ = p 2 ~ 
and 

V(X[' +1 ' > = q \ x[ 1 ^ = p) = G[p, q] for all i £ N and 

P(x¥ +1) = q | A 2 (i) =p) = G[p, q] for alii e N. 

We write X\ , X 2 for the sequences (A^ )* and (A 2 ^),- in the following. For q £ Q we regard r[q\ as a “reward” incurred 
when the chain is in state q. For each n £ N we define a random variable ll \ (n), the “accumulated reward before time n”: 

n—1 

Rx(n) 

i=0 

We define R 2 («) similarly, replacing A}’ by A 2 . Writing £ for expectation, we have for all n £ N: 

£Ri(n) = f(n)[pi] and £R 2 {n) = f{n)[p 2 ] (18) 

We now refine the definition of Xi and X 2 by coupling them as follows. Let s £ Q be a state from the only BSCC of G. Let 
T \, T 2 £ N so that: 

T\ := min{i e N | A^ = s} 

T 2 := min{i £ N | A 2 ^ = s} 

Note that 7], T 2 exist almost surely. We now require from X\ , X> that for each 1 £ N we have: 

• if there is j £ N with i — j = T 2 > T\, then A^ = Aj 7 ' 1 ^-*; 

• if there is j £ N with i — j = T\ > T 2 , then A= A 2 

In words: if Xi reaches s first, then as soon X 2 also reaches s, it mimics the behavior of X t after it had reached s; symmetrically, 
if X 2 reaches s first, then X t mimics X 2 in a similar way; if they reach s at the same time, their behavior is henceforth identical. 
Note that although Xi , X 2 are not independent. Equations ( IT71 ) and ( | I Sb remain valid, as they did not require independence. 
By the coupling we have: 

RiiTi + n) — i?i(Ti) = R 2 (T 2 + n) — R 2 {T 2 ) for all n £ N 

Let Ti <T 2 . Then it follows for all n £ N: 

R\ (Ti + n) — R 2 (T\ + n) = R 2 (T 2 + n) — R 2 (T 2 ) + Ri(Ti) — R 2 (Ti + n) 

< (T 2 — Ti)||r| + Ti||r| < {T\ + T 2 )|||r||| 

Let now T 2 < 7\. Then we similarly have for all ri £ N: 

R\(T\ + n) — R 2 (T\ + n) = R 2 {T 2 + n) — R 2 (T 2 ) + R\(T \) — R 2 {T\ + n) 

<i?i(T’ 1 )<T 1 ||r||| 


So in any case (T x < T 2 or T 2 < Ti) we have: 

Ri{n) - R 2 (n) < (T x + T 2 )|||r||| for all n £ N 


(19) 


We have: 


f{n)[pi] - f{n)[p 2 ] = £Ri(n) - £R 2 {n) 
= £ {Ri(n ) - R 2 (n)) 
<|||r||| (£T 1+ £T 2 ) 

< 10||r|||Q| /yM n 

= 10 \ r \diff IQI/Vmin 

The statement follows, as pi,p 2 were chosen arbitrarily. 


by (H 

by linearity of expectation 
by (QH) 
by Lemma [28] 
by (fl6ll 
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H. Bounded Differences 

We are going to prove Lemma I2T1 stating that the martingale defined in 0} has bounded differences. Recall that we have 
fixed a region R of A 2 whose type is II and which is determined by some p £ .S', where S satisfies is (2) < 0. Let us recall 
from 13 the definition of the function g : N — > referred to in ijT}. A certain vector g(fi) £ was defined in (7), and 

since the matrix A in 13 is stochastic, we can assume g( 0) > 0. Using g(0 ) as base case, we define the function g : N —>■ R s 
inductively as follows: 

g(n + 1) = rj. + Gg(n) for all n £ N, (20) 

where £ R s was defined in Q, and G £ R SxS denotes the matrix such that G[q,r ] is the probability that starting from 
q(l) the configuration r(0) is visited before visiting any configuration r'(0) for any r' ^ r. Since ig(2) < 0, the matrix G is 
stochastic, i.e., G1 = 1. We prove: 


Lemma 30. Matrix G has only one BSCC. 

k 

Proof: For p,q £ S and fcgZwe write p —» q if there is a path in A 2 from p(n ) to q(n + k) for some n > 0. Observe 

k 

that there is k < 0 with p —> q if and only if there is a nonempty path in the graph of G from p to q. 

Let p £ S and q £ B, where B C S is a BSCC of G. We need to show that q is reachable from p in the graph of G. Let 

k t 

k £ Z with p —> q. Since q is in the BSCC B of G, we have q —> q for some t < 0. By combining the two paths, we get 

j f k~\~CL& TYL 

p - > q and, by pumping the second path, p ->■ q for all a > 0. So by choosing a large enough we get p —>■ q for some 

m < 0. Hence there is a nonempty path in the graph of G from p to q. ■ 


Lemma 31. Let g max > 0 so that 0 < g{ 0) < g max 1. Let r max > 1 such that |||r.(J < r max . Let :r m j n be the smallest nonzero 
probability in the description of _4 2 - Then 

\m^ +1) - m^ ] \ < 2 + 2g max + 30\S\r max /x^l n for every l £ N. 

Proof: By (l20l >. using straightforward induction, we obtain for all n > 0: 

n— 1 

g{n) = G n g(f) +Y.G 1 (21) 

<9^1 4=0 <^ll 

Since G is stochastic, it follows for all n > 0: 

\\g(n + 1) - g(n)\\ < |||G" +1 5(0) - G"5(0)|| + ||G ri r 4 .||| < gmax + r m ax (22) 

For a vector v £ R s let us define 

M diff ~ mzx\v\p] - v[q)\ 

Observe that for any u,v £ R s we have |m + v\ di jj < |it| di ^ + \v\ di g. Let G be the constant from Lemma l29l for the matrix G. 
We have for all n > 0: 


\9{ n )\diff — \G n g{0)\diff 


A. gmax “t“ G \v | 


E G ‘n 


2=0 


by (ED 


diff 


A diff 
gmax "F t 2‘Gv rnax 


by Lemmas [29] and [ 

as \r\\ diff < 2r max 


Recall from ([]} that = xf' — trU + g{x.p) [pW] for all (el Hence we have: 


(th 


(^ +1 1 — m ^' 1 1 < \ x [ e ’ +1 ' > — x \ tS> 


— 1*^1 


+ l T fll 


+ 

<1 + 1 + 

+ 

<2 + 


9(x2 +1) )\p (e+1) } ~g(x ( 2 n )[p (l:) } 

g(4 w) )\p lt+1) ]-g(^) b (w) ] 

g(x i 2 ) )\p < ' e+1) ] - g(x { 2 ] )\p w ] 

9(x ( 2 +1) ) - g{x { 2 ] ) + g{x ( 2 ] ) 

<2 + 2 g 

max + _(2G + l)r. 

<3 c 


diff 


by ([23 and (E) 


(23) 
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|C| _ |C|3 

We have C := lQ\S\/y' m \ n , where y m i n is the smallest nonzero entry of G. By [TT] Corollary 6] we have y min > x[ nin , so 
we have C < 10|iS|/a:^ n . Hence: 

|m (£+1) - m w | < 2 + 2 g max + 30|5'|r maa: /a:lf i l I1 

■ 

Now we can prove Lemma |2TI 

Lemma |2l] There is a bound B > 1 computable in polynomial space such that — m^| < B for every l £ N. 

Proof: This follows from Lemma I3T1 as the vectors g{ 0) and rj., as defined in (7), can easily be expressed in the existential 
theory of the reals. ■ 
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